-- Logs begin at Wed 2022-11-02 06:44:09 UTC, end at Wed 2022-11-02 07:28:09 UTC. -- Nov 02 07:13:08.636022 dv-cinder-804035-51-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Nov 02 07:13:08.647452 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Nov 02 07:13:08.649518 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Nov 2 07:13:08 2022] *** Nov 02 07:13:08.649518 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Nov 02 07:13:08.649518 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Nov 02 07:13:08.649518 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: nodename: dv-cinder-804035-51-iscsi Nov 02 07:13:08.649518 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: machine: x86_64 Nov 02 07:13:08.649518 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: clock source: unix Nov 02 07:13:08.649518 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: pcre jit disabled Nov 02 07:13:08.649518 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: detected number of CPU cores: 4 Nov 02 07:13:08.649518 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: current working directory: / Nov 02 07:13:08.649518 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: detected binary path: /usr/bin/uwsgi-core Nov 02 07:13:08.649518 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: your processes number limit is 47680 Nov 02 07:13:08.650098 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: your memory page size is 4096 bytes Nov 02 07:13:08.650098 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: detected max file descriptor number: 2048 Nov 02 07:13:08.650098 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: lock engine: pthread robust mutexes Nov 02 07:13:08.650098 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: thunder lock: enabled Nov 02 07:13:08.650098 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Nov 02 07:13:08.650098 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Nov 02 07:13:08.688387 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: Python main interpreter initialized at 0x5563b9a2b9c0 Nov 02 07:13:08.688387 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: python threads support enabled Nov 02 07:13:08.688387 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: your server socket listen backlog is limited to 100 connections Nov 02 07:13:08.688387 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: your mercy for graceful operations on workers is 90 seconds Nov 02 07:13:08.688735 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: mapped 403077 bytes (393 KB) for 2 cores Nov 02 07:13:08.688822 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: *** Operational MODE: preforking *** Nov 02 07:13:08.689101 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: *** uWSGI is running in multiple interpreter mode *** Nov 02 07:13:08.689101 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: spawned uWSGI master process (pid: 81387) Nov 02 07:13:08.689290 dv-cinder-804035-51-iscsi systemd[1]: Started Devstack devstack@keystone.service. Nov 02 07:13:08.690027 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: spawned uWSGI worker 1 (pid: 81399, cores: 1) Nov 02 07:13:08.691218 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: spawned uWSGI worker 2 (pid: 81400, cores: 1) Nov 02 07:13:08.691218 dv-cinder-804035-51-iscsi devstack@keystone.service[81387]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Nov 02 07:13:12.953305 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:13:12.953305 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: from cryptography.utils import int_from_bytes Nov 02 07:13:12.954067 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:13:12.954067 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: from cryptography.utils import int_from_bytes Nov 02 07:13:12.954488 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:13:12.954488 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: from cryptography.utils import int_from_bytes Nov 02 07:13:12.954601 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:13:12.954601 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: from cryptography.utils import int_from_bytes Nov 02 07:13:13.004553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 07:13:13.004726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 07:13:13.004802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 07:13:13.004890 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 07:13:13.004967 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] command line args: [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 07:13:13.005041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] command line args: [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 07:13:13.005176 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 07:13:13.005264 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 07:13:13.005341 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ================================================================================ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 07:13:13.005448 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ================================================================================ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 07:13:13.005593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] admin_token = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.005710 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] admin_token = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.005840 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_dir = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.005963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_dir = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.006047 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.006156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.006238 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_source = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.006345 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_source = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.006426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] debug = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.006530 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] debug = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.006613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.006834 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.006958 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.007065 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.007160 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.007258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.007358 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.007457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.007551 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.007655 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.007765 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.007866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.007959 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_config_append = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.008057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_config_append = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.008149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.008258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.008359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_dir = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.008458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_dir = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.008575 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.008685 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.008797 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.008888 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.008960 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.009080 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.009153 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.009265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.009397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.009487 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.009579 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.009657 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.009771 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.009851 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.009977 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.010052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.010222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.010345 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.010466 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.010583 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.010717 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.010824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.010936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.011046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.011162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.011272 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.011402 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.011513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.011623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.011733 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.011844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.011962 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.012323 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.012446 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.012578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] publish_errors = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.012704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] publish_errors = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.012824 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.012941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.013053 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.013169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.013289 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.013406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.013517 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.013644 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.013764 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.013876 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.013992 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] standard_threads = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.014105 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] standard_threads = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.014221 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.014338 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.014457 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.014568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.014679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.014794 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.014906 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_journal = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.015017 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_journal = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.015192 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_json = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.015313 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_json = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.015425 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_stderr = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.015536 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_stderr = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.015647 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_syslog = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.015766 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_syslog = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.015889 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.016002 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:13:13.016112 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.016224 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.016340 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.016458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.016595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.016721 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.016834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.016944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.017054 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.017164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.017284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.017396 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.017506 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.017615 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.017731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.017848 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.017968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.018080 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.018190 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.018302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.019213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.019332 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.019453 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.019571 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.019683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.019794 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.019913 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.020034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.020154 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.020272 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.020391 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.020503 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.020642 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.020756 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.020877 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.020997 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.021109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.021219 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.021335 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.021449 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.021636 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.021749 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.021860 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.021970 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.022086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.022210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.022328 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.022446 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.022564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.022676 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.022787 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.022906 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.023019 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.023129 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.023239 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.023350 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.023466 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.023590 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.023703 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.023813 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.023930 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.024042 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.024153 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.024275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.024388 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.024499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.024635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.024748 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.024859 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.024977 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.025089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.025199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.025315 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.025426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.025536 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.025655 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.025767 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.025877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.025993 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.026106 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.026215 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.026338 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.026451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.026562 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.026672 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.026782 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.026898 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.027083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.027198 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.027315 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.027433 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.027547 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.027659 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.027785 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.027904 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.028016 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.028132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.028249 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.028361 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.028480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.028617 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.028715 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.028815 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.028916 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.029012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.029116 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.029212 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.029305 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.029399 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.029493 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.029598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.029710 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.029823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.029934 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.030078 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.030200 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.030311 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.030423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.030534 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.030645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.030756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.030874 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.030994 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.031098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.031209 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.031329 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.031440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.031554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.031673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.031794 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.031923 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.032033 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.032144 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.032262 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.032375 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.032487 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.032670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.032776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.032893 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.033007 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.033109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.033219 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.033339 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.033452 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.033564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.033676 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.033787 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.033899 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.034119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.034765 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.034881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.034993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.035106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.035218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.035343 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.035464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.035577 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.035737 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.035852 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.035965 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.036085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.036201 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.036313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.036430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.036556 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.036670 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.036794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.036908 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.037020 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.037138 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.037252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.037363 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.037483 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.037596 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.037708 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.037820 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.037931 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.038042 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.038161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.038276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.038393 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.038506 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.038618 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.038729 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.038914 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.039030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.039142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.039142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.039365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.039490 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.039604 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.039723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.039837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.039948 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.040060 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.040171 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.040283 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.040403 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.040517 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.040644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.040755 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.040866 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.040977 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.041103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.041217 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.041328 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.041440 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.041557 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.041676 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.041799 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.041913 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.042024 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.042135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.042277 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.external = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.042400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.042522 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.042636 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.external = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.042756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.042871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.042983 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.043095 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.043208 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.password = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.043319 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.043439 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.token = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.043553 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.password = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.043673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.043786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.token = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.043898 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.044010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.044130 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.044244 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.044356 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.044468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.044682 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.044802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.044928 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.045044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.045157 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.045268 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.045380 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.045493 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.045614 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.045729 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.045841 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.045954 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.046067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.046179 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.046300 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.046424 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.046538 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.046651 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.046763 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.046875 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.046999 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.047114 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.047225 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.047338 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.047450 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.047720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.047844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.047960 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.048104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.048219 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.048332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.048445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.048585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.048703 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.048816 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.048935 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.049068 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.049190 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.049312 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.049433 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.049547 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.049694 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.049813 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.049925 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.050046 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.050167 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.050281 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.050393 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.050506 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.050619 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.050742 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.050859 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.050972 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.051085 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.051198 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.051318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.051433 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.051546 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.051658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.051778 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.051894 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.052014 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.052129 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.052241 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.052363 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.052477 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.052603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.052794 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.052913 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.053026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.053139 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.053252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.053366 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.053487 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.053602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.053715 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.053828 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.053949 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.054065 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.054185 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.054300 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.054413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.054531 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.054646 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.054764 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.054886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.055001 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.055114 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.055226 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.055339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.055451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.055571 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.055686 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.055798 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.055917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.056032 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.056144 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.056266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.056384 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.056498 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.056623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.056736 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.056849 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.056973 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.057089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.057201 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.057314 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.057427 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.057539 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.057660 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.057816 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.057929 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.058041 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.058154 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.058266 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.058385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.058499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.058612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.058724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.058836 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.058949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.059069 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.059192 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.059316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.059431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.059543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.059663 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.059785 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.059899 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.060011 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.060123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.060237 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.060349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.060469 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.060608 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.060723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.060836 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.060949 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.061063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.061183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.061299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.061412 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.061531 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.061644 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.061764 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.061878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.061991 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.062110 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.062223 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.062338 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.062458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.062572 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.062685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.062797 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.062909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.063022 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.063146 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.063261 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.063374 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.063486 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.063599 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.063710 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.063830 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.063944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.064056 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.064169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.064280 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.064393 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.064513 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.064655 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.064769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.064882 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.064994 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.065106 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.065230 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.065380 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.065495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.065612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.065725 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.065837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.065957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.066072 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.066184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.066296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.066411 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.066526 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.066646 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.066761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.066873 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.066984 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.067096 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.067208 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.067327 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.067442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.067555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.067667 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.067785 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.067900 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.068020 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.068134 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.068247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.068359 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.068470 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.068629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.068757 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.068873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.068985 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.069098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.069211 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.069323 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.069444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.069558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.069672 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.069784 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.069903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.070017 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.070137 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.070252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.070364 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.070477 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.070590 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.070702 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.070822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.070943 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.071057 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.071170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.071282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.071395 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.071515 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.071630 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.071742 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.071855 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.071967 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.072079 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.072200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.072200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.072413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.072413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.072639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.072753 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.072878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.072994 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.073106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.073106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.073317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.073430 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.073550 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.073550 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.073770 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.073770 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.073983 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.074095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.074216 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.074330 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.074442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.074555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.074667 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.074779 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.074965 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.075081 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.075193 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.075306 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.075419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.075539 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.075654 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.075767 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.075880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.076023 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.076138 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.076258 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.076375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.076488 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.076711 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.076828 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.076941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.077067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.077183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.077297 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.077410 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.077522 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.077635 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.077756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.077871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.077983 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.078097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.078209 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.078322 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.078443 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.078558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.078558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.078770 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.078883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.078996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.078996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.079219 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.079332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.079445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.079557 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.079670 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.079791 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.079906 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.080018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.080151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.080267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.080388 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.080502 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.080635 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.080747 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.080860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.081502 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.081628 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.081744 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.081856 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.081968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.082081 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.082193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.082321 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.082436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.082549 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.082670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.082787 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.082900 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.083021 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.083136 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.083248 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.083361 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.083473 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.083585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.083705 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.083820 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.083932 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.084044 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.084157 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.084268 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.084390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.084504 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.084952 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.085066 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.085179 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.085291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.085417 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.085533 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.085645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.085757 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.085869 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.085981 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.086102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.086216 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.086329 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.086442 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.086554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.086666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.086786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.086903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.087017 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.087129 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.087241 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.087352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.087477 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.087603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.087717 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.087830 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.087942 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.088063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.088189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.088304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.088417 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.088554 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.088670 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.088783 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.088895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.089007 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.089120 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.089244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.089360 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.089472 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.089586 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.089698 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.089811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.089931 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.090046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.090159 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.090272 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.090385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.090498 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.090619 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.090734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.090847 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.090959 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.091071 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.091183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.091303 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.091418 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.091521 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.091633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.091747 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.091860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.091981 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.092096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.092209 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.092209 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 07:13:13.092421 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.092421 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.092421 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:13:13.092421 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 07:13:13.119844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.119999 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.120170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.120275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.120451 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.120785 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.129074 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.129625 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.157881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.158112 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.158563 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.158797 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.210319 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.210705 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.258669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.258903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.259799 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.260040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.411269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.411479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.411702 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.411921 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.412000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.412138 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.412229 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.412354 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.412435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.412595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.412817 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.413034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.435084 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.435230 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.435315 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.435468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.435550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.435700 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.435783 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.435921 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.436001 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.436132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.436222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.436346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.436432 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.436581 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.447860 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.448233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.460937 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.461046 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.461123 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.461195 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.461337 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.461420 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.461564 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.461645 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.461788 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.461870 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.461998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.462079 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.462212 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.462293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:13:13.479437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.479652 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:13:13.479737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.479926 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:13:13.480006 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.480191 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:13:13.480271 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.480456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:13:13.480559 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.480754 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.480834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.481015 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.481148 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.481276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.481535 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.481802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.490938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.491219 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.491317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.491492 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.491572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.491770 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.491849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.492055 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.498794 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.499061 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.501743 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.502002 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.502095 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.502291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.502376 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.502585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.502672 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.502892 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.502987 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.503180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.503322 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.503442 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.503605 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.503715 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.503887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.503990 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.504167 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.504473 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.504585 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.504789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.504994 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.505073 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.528106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.529602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.529839 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.529922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.531322 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.531613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.536924 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.537213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.537499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.537581 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.538010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.538315 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.543995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.544288 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.545629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.545926 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.548477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.548803 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.549105 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.549393 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.549691 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.549982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.550265 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.550549 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.551799 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.552101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.552393 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.552720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.553008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.553296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.553574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.553850 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.566158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.566439 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.569397 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.569681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.570313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.573519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.573604 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.573864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.576757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.577061 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.578100 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.581765 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.581869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.585306 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.585398 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.585610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.585901 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.586198 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.586489 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.586781 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.588645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.588954 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.589247 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.589540 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.589834 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.590134 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.603107 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.603398 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.603668 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.603933 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.606784 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.607080 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.607354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.607625 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.611107 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.612706 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.614273 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.614553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.614669 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.616104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.617209 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.617500 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.617704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.617890 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.618035 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.618180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.618482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.620711 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.621006 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.621296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.621597 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.622004 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.629664 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.629966 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.630347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.630635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.630940 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.631229 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.631510 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.633210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.633509 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.633805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.634091 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.634495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.634780 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.635075 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.647484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.650419 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.651038 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.653414 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.654007 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.654863 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.656489 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.656804 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.657008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.657089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.658458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.660510 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.660819 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.661229 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.663984 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.665506 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.667629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.668500 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.668821 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.669112 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.669192 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.669396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.672027 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.672427 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.672751 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.673041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.677246 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.677540 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.677826 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.680937 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.681230 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.681514 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.685647 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.685939 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.686225 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.686505 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.686781 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.687056 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.687337 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.687634 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.687909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.688228 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.688435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:13:13.688515 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.688751 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.688847 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.689039 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.689120 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.689301 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.689388 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.689591 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.689869 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.690149 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.690471 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.690882 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.691173 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.691452 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.691722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.692003 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.692278 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:13:13.723596 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5563b9a2b9c0 pid: 81399 (default app) Nov 02 07:13:13.726818 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29bb25-3d2b-4caa-94d0-f97325f3e32e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:13.727108 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29bb25-3d2b-4caa-94d0-f97325f3e32e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:13.727382 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29bb25-3d2b-4caa-94d0-f97325f3e32e None None] PATH_INFO: `/v3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:13.727579 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5563b9a2b9c0 pid: 81400 (default app) Nov 02 07:13:13.731527 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Wed Nov 2 07:13:13 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:20.451142 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85432973-8e4d-488d-89f5-ff9bfdcf9a80 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:20.451674 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85432973-8e4d-488d-89f5-ff9bfdcf9a80 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:20.451674 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85432973-8e4d-488d-89f5-ff9bfdcf9a80 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:20.455911 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:13:20.461812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefae43-e193-44db-a38a-e1590dc14623 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:20.462399 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefae43-e193-44db-a38a-e1590dc14623 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:20.462399 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefae43-e193-44db-a38a-e1590dc14623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:20.532734 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: INFO dbcounter [None req-7eefae43-e193-44db-a38a-e1590dc14623 None None] Registered counter for database keystone Nov 02 07:13:20.552731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG oslo_db.sqlalchemy.engines [None req-7eefae43-e193-44db-a38a-e1590dc14623 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81399) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 07:13:20.558839 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writer thread running {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 07:13:20.734917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG passlib.handlers.bcrypt [None req-7eefae43-e193-44db-a38a-e1590dc14623 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81399) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 07:13:20.735233 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG passlib.handlers.bcrypt [None req-7eefae43-e193-44db-a38a-e1590dc14623 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81399) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 07:13:20.792481 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7eefae43-e193-44db-a38a-e1590dc14623 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:20.794085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7eefae43-e193-44db-a38a-e1590dc14623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:20.920670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:13:21.392585 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85432973-8e4d-488d-89f5-ff9bfdcf9a80 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:21.462346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: INFO dbcounter [None req-85432973-8e4d-488d-89f5-ff9bfdcf9a80 None None] Registered counter for database keystone Nov 02 07:13:21.483340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG oslo_db.sqlalchemy.engines [None req-85432973-8e4d-488d-89f5-ff9bfdcf9a80 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81400) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 07:13:21.489597 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writer thread running {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 07:13:21.606411 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85432973-8e4d-488d-89f5-ff9bfdcf9a80 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:21.610622 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf2cbd-2f59-4858-9c7e-79e83738c643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:21.611241 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf2cbd-2f59-4858-9c7e-79e83738c643 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:21.611519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf2cbd-2f59-4858-9c7e-79e83738c643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:21.611791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf2cbd-2f59-4858-9c7e-79e83738c643 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:21.619916 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf2cbd-2f59-4858-9c7e-79e83738c643 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:21.671897 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: WARNING keystone.common.rbac_enforcer.enforcer [None req-98cf2cbd-2f59-4858-9c7e-79e83738c643 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 07:13:21.672248 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf2cbd-2f59-4858-9c7e-79e83738c643 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:21.681857 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Nov 2 07:13:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:13:21.687693 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefae43-e193-44db-a38a-e1590dc14623 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:21.705351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefae43-e193-44db-a38a-e1590dc14623 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:21.710506 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b6cb3-71bb-4486-b3b9-6413ba4b551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:21.711596 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b6cb3-71bb-4486-b3b9-6413ba4b551a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:21.711877 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b6cb3-71bb-4486-b3b9-6413ba4b551a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:21.712154 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b6cb3-71bb-4486-b3b9-6413ba4b551a admin admin] PATH_INFO: `/v3/projects/316fe9e17d5440ea82f955c062d5fbe0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:21.715903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b6cb3-71bb-4486-b3b9-6413ba4b551a admin admin] RBAC: Authorizing `identity:get_project(project_id=316fe9e17d5440ea82f955c062d5fbe0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:21.766589 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8d2b6cb3-71bb-4486-b3b9-6413ba4b551a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 07:13:21.767248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b6cb3-71bb-4486-b3b9-6413ba4b551a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:21.771706 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Wed Nov 2 07:13:21 2022] GET /identity/v3/projects/316fe9e17d5440ea82f955c062d5fbe0 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:23.876226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d068d2-daa8-44a4-a753-34665e79aa8b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:23.876821 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d068d2-daa8-44a4-a753-34665e79aa8b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:23.876821 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d068d2-daa8-44a4-a753-34665e79aa8b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:23.881439 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:23.886798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9b168-83e4-4b24-9d25-62692d3775e9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:23.887228 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9b168-83e4-4b24-9d25-62692d3775e9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:23.887345 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9b168-83e4-4b24-9d25-62692d3775e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:23.907809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-57c9b168-83e4-4b24-9d25-62692d3775e9 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:23.909309 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-57c9b168-83e4-4b24-9d25-62692d3775e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:23.917088 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 07:13:24.184827 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d068d2-daa8-44a4-a753-34665e79aa8b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:24.197320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d068d2-daa8-44a4-a753-34665e79aa8b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:24.201519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd4895-db9f-437f-9a0a-68967ecbcaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:24.202120 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd4895-db9f-437f-9a0a-68967ecbcaee admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:24.202403 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd4895-db9f-437f-9a0a-68967ecbcaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:24.202680 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd4895-db9f-437f-9a0a-68967ecbcaee admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:24.210504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd4895-db9f-437f-9a0a-68967ecbcaee admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:24.212272 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd4895-db9f-437f-9a0a-68967ecbcaee admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:24.220932 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Nov 2 07:13:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:13:24.226020 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9b168-83e4-4b24-9d25-62692d3775e9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:24.234914 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9b168-83e4-4b24-9d25-62692d3775e9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:24.239745 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642f5d7-01dc-4673-8823-46d79ec0be6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:24.240369 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642f5d7-01dc-4673-8823-46d79ec0be6f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:24.240679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642f5d7-01dc-4673-8823-46d79ec0be6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:24.240954 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642f5d7-01dc-4673-8823-46d79ec0be6f admin admin] PATH_INFO: `/v3/users/6ac58377d56b4184a21764af3c195e68` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:24.242259 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642f5d7-01dc-4673-8823-46d79ec0be6f admin admin] RBAC: Authorizing `identity:get_user(user_id=6ac58377d56b4184a21764af3c195e68)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:24.244068 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642f5d7-01dc-4673-8823-46d79ec0be6f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:24.245607 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Wed Nov 2 07:13:24 2022] GET /identity/v3/users/6ac58377d56b4184a21764af3c195e68 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:26.422254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d972766-1d19-417e-8ef4-5d80d7425af9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:26.422772 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d972766-1d19-417e-8ef4-5d80d7425af9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:26.422881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d972766-1d19-417e-8ef4-5d80d7425af9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:26.424138 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:26.429831 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710b891-7f88-4e99-9113-8a5bb13a8828 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:26.430250 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710b891-7f88-4e99-9113-8a5bb13a8828 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:26.430501 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710b891-7f88-4e99-9113-8a5bb13a8828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:26.448899 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d710b891-7f88-4e99-9113-8a5bb13a8828 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:26.450071 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d710b891-7f88-4e99-9113-8a5bb13a8828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:26.470515 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98323-3c15-4385-a197-b489997e4a6c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:26.470799 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98323-3c15-4385-a197-b489997e4a6c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:26.471071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98323-3c15-4385-a197-b489997e4a6c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:26.472402 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:13:26.477538 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8c1a1-92a3-4991-b036-4be94abfa268 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:26.477819 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8c1a1-92a3-4991-b036-4be94abfa268 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:26.478094 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8c1a1-92a3-4991-b036-4be94abfa268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:26.520249 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG passlib.handlers.bcrypt [None req-33f8c1a1-92a3-4991-b036-4be94abfa268 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81400) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 07:13:26.520684 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG passlib.handlers.bcrypt [None req-33f8c1a1-92a3-4991-b036-4be94abfa268 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81400) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 07:13:26.531985 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:26.570133 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-33f8c1a1-92a3-4991-b036-4be94abfa268 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:26.571729 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-33f8c1a1-92a3-4991-b036-4be94abfa268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:26.671153 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:13:26.796081 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710b891-7f88-4e99-9113-8a5bb13a8828 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:26.819883 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:26.897670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710b891-7f88-4e99-9113-8a5bb13a8828 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:26.930681 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e61fe0-ac8a-4ec2-b32d-498b9fbb1d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:26.931312 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e61fe0-ac8a-4ec2-b32d-498b9fbb1d70 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:26.931593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e61fe0-ac8a-4ec2-b32d-498b9fbb1d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:26.931871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e61fe0-ac8a-4ec2-b32d-498b9fbb1d70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:26.936522 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8c1a1-92a3-4991-b036-4be94abfa268 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:26.943682 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e61fe0-ac8a-4ec2-b32d-498b9fbb1d70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-75e61fe0-ac8a-4ec2-b32d-498b9fbb1d70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:26.953713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:26.954683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:13:26.955658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:13:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:26.960760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e61fe0-ac8a-4ec2-b32d-498b9fbb1d70 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:27.038181 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8c1a1-92a3-4991-b036-4be94abfa268 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:27.040410 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:27.041398 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:27.060080 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e61fe0-ac8a-4ec2-b32d-498b9fbb1d70 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:27.075023 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608055b5-9d18-48a0-b70f-17d810f889d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:27.075893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608055b5-9d18-48a0-b70f-17d810f889d0 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:27.076180 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608055b5-9d18-48a0-b70f-17d810f889d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:27.076460 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608055b5-9d18-48a0-b70f-17d810f889d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:27.077456 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608055b5-9d18-48a0-b70f-17d810f889d0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:27.078887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608055b5-9d18-48a0-b70f-17d810f889d0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:27.090508 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.sql.core [None req-608055b5-9d18-48a0-b70f-17d810f889d0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:13:27.090508 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:13:27.090508 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [parameters: {'id': '56e927d85551490f8359a983884f6fe2', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:13:27.090508 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81400) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:13:27.093423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33737d21-31f2-4bec-ba7a-8eb96141613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:27.094039 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33737d21-31f2-4bec-ba7a-8eb96141613e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:27.094324 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33737d21-31f2-4bec-ba7a-8eb96141613e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:27.094599 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33737d21-31f2-4bec-ba7a-8eb96141613e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:27.095267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33737d21-31f2-4bec-ba7a-8eb96141613e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:27.095572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33737d21-31f2-4bec-ba7a-8eb96141613e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:27.097526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33737d21-31f2-4bec-ba7a-8eb96141613e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-608055b5-9d18-48a0-b70f-17d810f889d0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:13:27.109351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:13:27.110165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application yield resource Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise self.value Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application session.commit() Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:13:27.111040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:13:27.111885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:13:27.112860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:13:27.113599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [parameters: {'id': '56e927d85551490f8359a983884f6fe2', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:13:27.114368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:27.115101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:13:27.115964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:27.116574 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:27.116650 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 8/16] 10.222.0.15 () {60 vars in 1137 bytes} [Wed Nov 2 07:13:26 2022] POST /identity/v3/domains => generated 190 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:13:27.116900 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33737d21-31f2-4bec-ba7a-8eb96141613e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:27.118083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608055b5-9d18-48a0-b70f-17d810f889d0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:27.216483 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33737d21-31f2-4bec-ba7a-8eb96141613e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:27.217030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608055b5-9d18-48a0-b70f-17d810f889d0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:27.249894 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2b52d-ab7b-47f7-a906-60fef1992b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:27.250253 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240678d8-e60b-4faa-bb4c-db539197075d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:27.250536 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b52d-ab7b-47f7-a906-60fef1992b2b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:27.250820 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b52d-ab7b-47f7-a906-60fef1992b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:27.251094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b52d-ab7b-47f7-a906-60fef1992b2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:27.251370 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240678d8-e60b-4faa-bb4c-db539197075d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:27.251719 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240678d8-e60b-4faa-bb4c-db539197075d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:27.252003 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240678d8-e60b-4faa-bb4c-db539197075d None admin] PATH_INFO: `/v3/users/6ac58377d56b4184a21764af3c195e68` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:27.253414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240678d8-e60b-4faa-bb4c-db539197075d None admin] RBAC: Authorizing `identity:get_user(user_id=6ac58377d56b4184a21764af3c195e68)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:27.254874 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240678d8-e60b-4faa-bb4c-db539197075d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:27.256454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:27 2022] GET /identity/v3/users/6ac58377d56b4184a21764af3c195e68 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:27.261158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2b52d-ab7b-47f7-a906-60fef1992b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:27.261922 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240678d8-e60b-4faa-bb4c-db539197075d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:27.262609 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2b52d-ab7b-47f7-a906-60fef1992b2b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:27.265125 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0da2b52d-ab7b-47f7-a906-60fef1992b2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:13:27.280909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:27.281775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:13:27.282554 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:13:27.282554 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:13:27.282554 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:13:27.282554 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 9/18] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:13:27.288501 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2b52d-ab7b-47f7-a906-60fef1992b2b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:27.298109 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:27.363516 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240678d8-e60b-4faa-bb4c-db539197075d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:27.386185 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2b52d-ab7b-47f7-a906-60fef1992b2b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:27.396632 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d4756-e9b4-476b-905d-de6fc88f87cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:27.397259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d4756-e9b4-476b-905d-de6fc88f87cf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:27.397538 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d4756-e9b4-476b-905d-de6fc88f87cf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:27.397811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d4756-e9b4-476b-905d-de6fc88f87cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:27.398932 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d4756-e9b4-476b-905d-de6fc88f87cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:27.400382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d4756-e9b4-476b-905d-de6fc88f87cf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:27.401981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 10/19] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:27.407454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d4756-e9b4-476b-905d-de6fc88f87cf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:27.419092 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f42b1-8a65-4cbd-b6fc-8d750c468a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:27.419777 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f42b1-8a65-4cbd-b6fc-8d750c468a0b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:27.420063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f42b1-8a65-4cbd-b6fc-8d750c468a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:27.420339 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f42b1-8a65-4cbd-b6fc-8d750c468a0b None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:27.421007 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f42b1-8a65-4cbd-b6fc-8d750c468a0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:27.421311 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f42b1-8a65-4cbd-b6fc-8d750c468a0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:27.422687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f42b1-8a65-4cbd-b6fc-8d750c468a0b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:27.440105 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 10/20] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:27.510184 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d4756-e9b4-476b-905d-de6fc88f87cf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:27.512828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:27.548503 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426c934-c5a3-4fda-9e67-87c8318f172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:27.549204 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426c934-c5a3-4fda-9e67-87c8318f172f None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:27.549484 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426c934-c5a3-4fda-9e67-87c8318f172f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:27.549790 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426c934-c5a3-4fda-9e67-87c8318f172f None admin] PATH_INFO: `/v3/domains/default/users/6ac58377d56b4184a21764af3c195e68/roles/81d287a8b606458d87c9607e8b48ac58` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:27.552358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426c934-c5a3-4fda-9e67-87c8318f172f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6ac58377d56b4184a21764af3c195e68, role_id=81d287a8b606458d87c9607e8b48ac58)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:27.553855 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426c934-c5a3-4fda-9e67-87c8318f172f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:27.670778 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 11/21] 10.222.0.15 () {58 vars in 1361 bytes} [Wed Nov 2 07:13:27 2022] PUT /identity/v3/domains/default/users/6ac58377d56b4184a21764af3c195e68/roles/81d287a8b606458d87c9607e8b48ac58 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:13:29.976942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724953a-bf84-4997-9289-858bb4db2dce None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:29.977411 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724953a-bf84-4997-9289-858bb4db2dce None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:29.977411 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724953a-bf84-4997-9289-858bb4db2dce None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:29.978744 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 11/22] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:29.984574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943bfb6-12e6-4298-b5d4-ad3070cf0a16 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:29.985001 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943bfb6-12e6-4298-b5d4-ad3070cf0a16 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:29.985177 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943bfb6-12e6-4298-b5d4-ad3070cf0a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:30.009987 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4943bfb6-12e6-4298-b5d4-ad3070cf0a16 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:30.011604 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4943bfb6-12e6-4298-b5d4-ad3070cf0a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:30.093993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 12/23] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:30.161605 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbd22c-ab6b-4d83-b215-95bcc920a913 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:30.161903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbd22c-ab6b-4d83-b215-95bcc920a913 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:30.162161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbd22c-ab6b-4d83-b215-95bcc920a913 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:30.163819 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 12/24] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:30.169270 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156fd34-d940-451a-acdf-407c3ba87e38 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:30.169560 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156fd34-d940-451a-acdf-407c3ba87e38 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:30.169841 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156fd34-d940-451a-acdf-407c3ba87e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:30.192470 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a156fd34-d940-451a-acdf-407c3ba87e38 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:30.193698 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a156fd34-d940-451a-acdf-407c3ba87e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:30.283486 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 13/25] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:30.523177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d5dd5-b85e-4d3b-90e2-95446392eaa5 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:30.523532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d5dd5-b85e-4d3b-90e2-95446392eaa5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:30.523736 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d5dd5-b85e-4d3b-90e2-95446392eaa5 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:30.525182 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 13/26] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:30.526409 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156fd34-d940-451a-acdf-407c3ba87e38 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:30.530341 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a8d4b-82bb-4e2d-8649-0ba7f7433169 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:30.530843 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a8d4b-82bb-4e2d-8649-0ba7f7433169 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:30.531317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a8d4b-82bb-4e2d-8649-0ba7f7433169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:30.562422 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4b7a8d4b-82bb-4e2d-8649-0ba7f7433169 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:30.566934 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4b7a8d4b-82bb-4e2d-8649-0ba7f7433169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:30.589876 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:30.674784 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:30.705870 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 14/27] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:13:30.706461 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a8d4b-82bb-4e2d-8649-0ba7f7433169 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:30.749844 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156fd34-d940-451a-acdf-407c3ba87e38 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:30.845329 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259c955-a63a-4fa1-b9b2-705c174a02ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:30.845905 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259c955-a63a-4fa1-b9b2-705c174a02ca None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:30.846189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259c955-a63a-4fa1-b9b2-705c174a02ca None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:30.846465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259c955-a63a-4fa1-b9b2-705c174a02ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:30.849600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a8d4b-82bb-4e2d-8649-0ba7f7433169 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:30.881660 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259c955-a63a-4fa1-b9b2-705c174a02ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:30.883357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259c955-a63a-4fa1-b9b2-705c174a02ca None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:30.887069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990331df-2508-47e8-98b2-7038c40321c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:30.887686 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990331df-2508-47e8-98b2-7038c40321c7 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:30.887964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990331df-2508-47e8-98b2-7038c40321c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:30.888235 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990331df-2508-47e8-98b2-7038c40321c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:30.888930 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990331df-2508-47e8-98b2-7038c40321c7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:30.891109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990331df-2508-47e8-98b2-7038c40321c7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e259c955-a63a-4fa1-b9b2-705c174a02ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:13:30.910494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:30.911426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:13:30.912233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:13:30.912233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:13:30.912233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:30.914202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 14/28] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:30.915683 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc572-f485-426f-a4f3-78b6e8e409ba None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:30.920645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc572-f485-426f-a4f3-78b6e8e409ba None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:30.921278 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc572-f485-426f-a4f3-78b6e8e409ba None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:30.926122 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 15/29] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:30.927442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc572-f485-426f-a4f3-78b6e8e409ba None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:31.032261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 15/30] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Nov 2 07:13:30 2022] POST /identity/v3/roles => generated 219 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:13:31.034022 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff61a56-8ca2-4304-b98a-6dc49d57f59a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:31.034909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff61a56-8ca2-4304-b98a-6dc49d57f59a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:31.035186 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff61a56-8ca2-4304-b98a-6dc49d57f59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:31.059472 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8ff61a56-8ca2-4304-b98a-6dc49d57f59a None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:31.061117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8ff61a56-8ca2-4304-b98a-6dc49d57f59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:31.074064 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:31.118742 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:31.124321 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc572-f485-426f-a4f3-78b6e8e409ba None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:31.160202 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 16/31] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:31.202838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3779725-1bc4-49f1-a7fe-d03ea3f2ea39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:31.203477 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3779725-1bc4-49f1-a7fe-d03ea3f2ea39 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:31.203762 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3779725-1bc4-49f1-a7fe-d03ea3f2ea39 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:31.204040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3779725-1bc4-49f1-a7fe-d03ea3f2ea39 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:31.204739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3779725-1bc4-49f1-a7fe-d03ea3f2ea39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:31.205046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3779725-1bc4-49f1-a7fe-d03ea3f2ea39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:31.206531 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3779725-1bc4-49f1-a7fe-d03ea3f2ea39 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:31.235375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 16/32] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:31.241891 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff61a56-8ca2-4304-b98a-6dc49d57f59a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:31.287047 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497a1bd-fecb-4ccf-981e-b57f71bf4b56 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:31.287047 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497a1bd-fecb-4ccf-981e-b57f71bf4b56 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:31.287442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497a1bd-fecb-4ccf-981e-b57f71bf4b56 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:31.288991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 17/33] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:31.293987 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ff01-9539-4a6b-9baf-6820c73c0237 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:31.294280 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ff01-9539-4a6b-9baf-6820c73c0237 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:31.294559 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ff01-9539-4a6b-9baf-6820c73c0237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:31.317570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ac37ff01-9539-4a6b-9baf-6820c73c0237 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:31.319095 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ac37ff01-9539-4a6b-9baf-6820c73c0237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:31.374965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff61a56-8ca2-4304-b98a-6dc49d57f59a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:31.402240 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 18/34] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:31.403167 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37ff01-9539-4a6b-9baf-6820c73c0237 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:31.422313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d04ba-cb51-4f30-809c-acb3fe117390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:31.422940 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d04ba-cb51-4f30-809c-acb3fe117390 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:31.423222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d04ba-cb51-4f30-809c-acb3fe117390 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:31.423496 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d04ba-cb51-4f30-809c-acb3fe117390 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:31.424167 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d04ba-cb51-4f30-809c-acb3fe117390 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:31.426400 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d04ba-cb51-4f30-809c-acb3fe117390 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:31.474181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 17/35] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:13:31 2022] POST /identity/v3/projects => generated 301 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:13:31.508899 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:31.554206 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d04ba-cb51-4f30-809c-acb3fe117390 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:31.562707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37ff01-9539-4a6b-9baf-6820c73c0237 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:31.610976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56098a71-5da5-4eeb-bd0d-f02aaa9dfe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:31.611625 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56098a71-5da5-4eeb-bd0d-f02aaa9dfe2a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:31.611905 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56098a71-5da5-4eeb-bd0d-f02aaa9dfe2a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:31.612176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56098a71-5da5-4eeb-bd0d-f02aaa9dfe2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:31.619670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:31.627704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56098a71-5da5-4eeb-bd0d-f02aaa9dfe2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-56098a71-5da5-4eeb-bd0d-f02aaa9dfe2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:31.635699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:31.636645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:31.637385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 19/36] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:13:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:31.650870 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56098a71-5da5-4eeb-bd0d-f02aaa9dfe2a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:31.767436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d04ba-cb51-4f30-809c-acb3fe117390 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:31.775332 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56098a71-5da5-4eeb-bd0d-f02aaa9dfe2a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:31.805574 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3c3ee-73f2-4508-9d0b-0058d8f66965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:31.806285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c3ee-73f2-4508-9d0b-0058d8f66965 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:31.806675 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c3ee-73f2-4508-9d0b-0058d8f66965 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:31.807064 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c3ee-73f2-4508-9d0b-0058d8f66965 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:31.807507 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:31.808194 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3c3ee-73f2-4508-9d0b-0058d8f66965 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:31.809607 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3c3ee-73f2-4508-9d0b-0058d8f66965 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:31.821329 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124114b2-da25-4541-a6ad-9fb9db44129b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:31.822357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124114b2-da25-4541-a6ad-9fb9db44129b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:31.822701 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124114b2-da25-4541-a6ad-9fb9db44129b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:31.822986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124114b2-da25-4541-a6ad-9fb9db44129b None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:31.823678 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124114b2-da25-4541-a6ad-9fb9db44129b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:31.823980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124114b2-da25-4541-a6ad-9fb9db44129b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:31.825398 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124114b2-da25-4541-a6ad-9fb9db44129b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:31.825815 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 18/37] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Nov 2 07:13:31 2022] POST /identity/v3/roles => generated 225 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:13:31.826758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3c3ee-73f2-4508-9d0b-0058d8f66965 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:31.844744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 20/38] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:31.848220 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124114b2-da25-4541-a6ad-9fb9db44129b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:31.942979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:31.959304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3c3ee-73f2-4508-9d0b-0058d8f66965 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:31.973744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124114b2-da25-4541-a6ad-9fb9db44129b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:32.003768 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fadaf-b477-469e-bf82-99124e0dafc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:32.004846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fadaf-b477-469e-bf82-99124e0dafc1 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:32.005201 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fadaf-b477-469e-bf82-99124e0dafc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:32.005483 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fadaf-b477-469e-bf82-99124e0dafc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:32.006409 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fadaf-b477-469e-bf82-99124e0dafc1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:32.007983 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fadaf-b477-469e-bf82-99124e0dafc1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:32.015126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e47db-d3df-4116-a83b-cdf736920307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:32.015788 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e47db-d3df-4116-a83b-cdf736920307 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:32.016069 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e47db-d3df-4116-a83b-cdf736920307 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:32.016340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e47db-d3df-4116-a83b-cdf736920307 None admin] PATH_INFO: `/v3/users/6ac58377d56b4184a21764af3c195e68` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:32.018058 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e47db-d3df-4116-a83b-cdf736920307 None admin] RBAC: Authorizing `identity:get_user(user_id=6ac58377d56b4184a21764af3c195e68)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:32.019685 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e47db-d3df-4116-a83b-cdf736920307 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:32.021584 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 21/39] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:31 2022] GET /identity/v3/users/6ac58377d56b4184a21764af3c195e68 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:32.027804 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e47db-d3df-4116-a83b-cdf736920307 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:32.036463 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 19/40] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Nov 2 07:13:31 2022] POST /identity/v3/roles => generated 223 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:13:32.099751 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:32.116495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bd3cc-543c-4241-ada6-96db4d74842b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:32.116820 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bd3cc-543c-4241-ada6-96db4d74842b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:32.117103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bd3cc-543c-4241-ada6-96db4d74842b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:32.118490 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 20/41] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:13:32.124239 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0c852-7ab8-42b7-8d3f-bd5886571475 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:32.124528 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0c852-7ab8-42b7-8d3f-bd5886571475 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:32.124820 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0c852-7ab8-42b7-8d3f-bd5886571475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:32.147144 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-73e0c852-7ab8-42b7-8d3f-bd5886571475 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:32.147144 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-73e0c852-7ab8-42b7-8d3f-bd5886571475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:32.159866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e47db-d3df-4116-a83b-cdf736920307 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:32.202637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e795ed-1d8a-4b2a-88ce-0e9b79984e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:32.203208 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e795ed-1d8a-4b2a-88ce-0e9b79984e0b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:32.203515 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e795ed-1d8a-4b2a-88ce-0e9b79984e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:32.204631 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e795ed-1d8a-4b2a-88ce-0e9b79984e0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:32.206213 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e795ed-1d8a-4b2a-88ce-0e9b79984e0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:32.206975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e795ed-1d8a-4b2a-88ce-0e9b79984e0b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:32.208900 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 22/42] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:32.214797 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e795ed-1d8a-4b2a-88ce-0e9b79984e0b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:32.246287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 21/43] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:13:32.321983 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e795ed-1d8a-4b2a-88ce-0e9b79984e0b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:32.344325 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:32.354956 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716807d-f8aa-49a0-bfc4-cb3320305378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:32.355564 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716807d-f8aa-49a0-bfc4-cb3320305378 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:32.355844 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716807d-f8aa-49a0-bfc4-cb3320305378 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:32.356118 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716807d-f8aa-49a0-bfc4-cb3320305378 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:32.356878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716807d-f8aa-49a0-bfc4-cb3320305378 None admin] RBAC: Adding query filter params (role.id=81d287a8b606458d87c9607e8b48ac58, scope.domain.id=default, user.id=6ac58377d56b4184a21764af3c195e68) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:32.357186 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716807d-f8aa-49a0-bfc4-cb3320305378 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:32.358613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716807d-f8aa-49a0-bfc4-cb3320305378 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:32.366787 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 23/44] 10.222.0.15 () {56 vars in 1322 bytes} [Wed Nov 2 07:13:32 2022] GET /identity/v3/role_assignments?user.id=6ac58377d56b4184a21764af3c195e68&scope.domain.id=default&role.id=81d287a8b606458d87c9607e8b48ac58 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:32.515501 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0c852-7ab8-42b7-8d3f-bd5886571475 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:32.596131 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:32.618414 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0c852-7ab8-42b7-8d3f-bd5886571475 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:32.651857 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432259f-c01d-4845-925b-c64da4ef1e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:32.652497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432259f-c01d-4845-925b-c64da4ef1e26 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:32.652809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432259f-c01d-4845-925b-c64da4ef1e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:32.653094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432259f-c01d-4845-925b-c64da4ef1e26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:32.654184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432259f-c01d-4845-925b-c64da4ef1e26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:32.655636 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432259f-c01d-4845-925b-c64da4ef1e26 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:32.657218 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 22/45] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:32.662880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716807d-f8aa-49a0-bfc4-cb3320305378 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:32.759572 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716807d-f8aa-49a0-bfc4-cb3320305378 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:32.791876 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec6c18-7f73-40a5-8916-45216ca061ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:32.792420 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6c18-7f73-40a5-8916-45216ca061ba None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:32.792731 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6c18-7f73-40a5-8916-45216ca061ba None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:32.793005 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6c18-7f73-40a5-8916-45216ca061ba None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:32.793663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec6c18-7f73-40a5-8916-45216ca061ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:32.795090 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec6c18-7f73-40a5-8916-45216ca061ba None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:32.821665 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 24/46] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:13:32 2022] POST /identity/v3/projects => generated 312 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:13:34.934078 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f746ef-1704-49dc-b3f0-1be4266b7a81 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:34.934557 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f746ef-1704-49dc-b3f0-1be4266b7a81 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:34.934557 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f746ef-1704-49dc-b3f0-1be4266b7a81 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:34.935906 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 23/47] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:34.941641 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87919c76-6b3b-4bfd-ba17-31f551ac2bdc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:34.942065 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87919c76-6b3b-4bfd-ba17-31f551ac2bdc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:34.942197 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87919c76-6b3b-4bfd-ba17-31f551ac2bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:34.960614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-87919c76-6b3b-4bfd-ba17-31f551ac2bdc None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:34.961803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-87919c76-6b3b-4bfd-ba17-31f551ac2bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:34.994402 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:35.027137 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 25/48] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:35.293795 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f746ef-1704-49dc-b3f0-1be4266b7a81 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:35.393373 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f746ef-1704-49dc-b3f0-1be4266b7a81 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:35.531181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38566083-d338-44bf-8daa-9cc078b6e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:35.531811 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38566083-d338-44bf-8daa-9cc078b6e0ec None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:35.532097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38566083-d338-44bf-8daa-9cc078b6e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:35.532374 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38566083-d338-44bf-8daa-9cc078b6e0ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:35.533517 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38566083-d338-44bf-8daa-9cc078b6e0ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:35.534965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38566083-d338-44bf-8daa-9cc078b6e0ec None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:35.536528 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 24/49] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:35.542303 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87919c76-6b3b-4bfd-ba17-31f551ac2bdc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:35.637668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87919c76-6b3b-4bfd-ba17-31f551ac2bdc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:35.671970 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6ca7e-18a2-4456-9a25-9560d3edf5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:35.672596 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6ca7e-18a2-4456-9a25-9560d3edf5b5 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:35.672878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6ca7e-18a2-4456-9a25-9560d3edf5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:35.673146 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6ca7e-18a2-4456-9a25-9560d3edf5b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:35.673792 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6ca7e-18a2-4456-9a25-9560d3edf5b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:35.675859 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6ca7e-18a2-4456-9a25-9560d3edf5b5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:35.700911 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 26/50] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Nov 2 07:13:35 2022] POST /identity/v3/projects => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:13:37.840858 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faf346-d8f7-4fa5-8b03-1bc5cdbd4d57 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:37.841351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faf346-d8f7-4fa5-8b03-1bc5cdbd4d57 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:37.841450 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faf346-d8f7-4fa5-8b03-1bc5cdbd4d57 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:37.842745 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 25/51] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:37.847912 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb20e2e-5f8c-4010-af38-e7965eb7782d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:37.848269 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb20e2e-5f8c-4010-af38-e7965eb7782d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:37.848580 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb20e2e-5f8c-4010-af38-e7965eb7782d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:37.969667 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4eb20e2e-5f8c-4010-af38-e7965eb7782d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:37.970808 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4eb20e2e-5f8c-4010-af38-e7965eb7782d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:37.987304 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:38.037522 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 27/52] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:38.305922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faf346-d8f7-4fa5-8b03-1bc5cdbd4d57 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:38.322220 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:38.404147 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faf346-d8f7-4fa5-8b03-1bc5cdbd4d57 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:38.435958 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1356db-8fcb-4261-9a55-6dda68112e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:38.436600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1356db-8fcb-4261-9a55-6dda68112e46 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:38.436882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1356db-8fcb-4261-9a55-6dda68112e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:38.437153 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1356db-8fcb-4261-9a55-6dda68112e46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:38.438243 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1356db-8fcb-4261-9a55-6dda68112e46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:38.439667 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1356db-8fcb-4261-9a55-6dda68112e46 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:38.441777 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 26/53] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:38.447324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb20e2e-5f8c-4010-af38-e7965eb7782d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:38.541823 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb20e2e-5f8c-4010-af38-e7965eb7782d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:38.574618 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57359bc0-91ab-4a71-a95b-fe0cc3423d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:38.575168 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57359bc0-91ab-4a71-a95b-fe0cc3423d4b None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:38.575450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57359bc0-91ab-4a71-a95b-fe0cc3423d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:38.575722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57359bc0-91ab-4a71-a95b-fe0cc3423d4b None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:38.576377 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57359bc0-91ab-4a71-a95b-fe0cc3423d4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:38.577843 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57359bc0-91ab-4a71-a95b-fe0cc3423d4b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:38.603839 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 28/54] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:13:38 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:13:41.819353 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9818a-add9-422d-87ef-3bc173a874b7 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:41.820313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9818a-add9-422d-87ef-3bc173a874b7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:41.820851 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9818a-add9-422d-87ef-3bc173a874b7 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:41.822703 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 27/55] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:41.827620 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c97efd-220a-4eda-b949-aa3dc3a71179 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:41.828403 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c97efd-220a-4eda-b949-aa3dc3a71179 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:41.828895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c97efd-220a-4eda-b949-aa3dc3a71179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:41.857499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-35c97efd-220a-4eda-b949-aa3dc3a71179 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:41.859067 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-35c97efd-220a-4eda-b949-aa3dc3a71179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:41.861381 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:41.913725 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a6dfb-2c19-461f-81c2-77adb612919d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:41.913930 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a6dfb-2c19-461f-81c2-77adb612919d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:41.914207 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a6dfb-2c19-461f-81c2-77adb612919d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:41.915731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 28/56] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:41.921905 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151bd5d-cdd9-46f4-b34a-461ce3e44a91 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:41.922410 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151bd5d-cdd9-46f4-b34a-461ce3e44a91 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:41.922933 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151bd5d-cdd9-46f4-b34a-461ce3e44a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:41.948009 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 29/57] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:13:41.951873 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3151bd5d-cdd9-46f4-b34a-461ce3e44a91 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:41.953083 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3151bd5d-cdd9-46f4-b34a-461ce3e44a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:42.050279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 29/58] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:13:42.214002 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb0756-ae8c-42a6-8363-f4d8c6c01602 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:42.214490 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb0756-ae8c-42a6-8363-f4d8c6c01602 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:42.214927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb0756-ae8c-42a6-8363-f4d8c6c01602 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:42.216440 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 30/59] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:42.233076 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370eb3-8294-4240-9715-e6890f3294d8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:42.233367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370eb3-8294-4240-9715-e6890f3294d8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:42.233644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370eb3-8294-4240-9715-e6890f3294d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:42.244354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb0756-ae8c-42a6-8363-f4d8c6c01602 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:42.271985 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a6370eb3-8294-4240-9715-e6890f3294d8 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:42.273221 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a6370eb3-8294-4240-9715-e6890f3294d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:42.281756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:42.345722 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 30/60] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:42.349710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7843f-e37d-4b6b-a0a6-ac616e0b61a6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:42.349710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7843f-e37d-4b6b-a0a6-ac616e0b61a6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:42.349710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7843f-e37d-4b6b-a0a6-ac616e0b61a6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:42.349710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 31/61] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:42.357797 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c7156-162e-4096-b7a7-728368091b32 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:42.358322 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c7156-162e-4096-b7a7-728368091b32 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:42.358918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c7156-162e-4096-b7a7-728368091b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:42.377507 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-091c7156-162e-4096-b7a7-728368091b32 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:42.380209 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-091c7156-162e-4096-b7a7-728368091b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:42.398975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb0756-ae8c-42a6-8363-f4d8c6c01602 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:42.448463 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fcf28-33a5-414c-8b03-e8c961251715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:42.449529 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fcf28-33a5-414c-8b03-e8c961251715 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:42.450008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fcf28-33a5-414c-8b03-e8c961251715 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:42.450484 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fcf28-33a5-414c-8b03-e8c961251715 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:42.457944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:42.463258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fcf28-33a5-414c-8b03-e8c961251715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-724fcf28-33a5-414c-8b03-e8c961251715 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:42.468664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:42.469465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:13:42.470218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:42.473345 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 31/62] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:42.474800 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fcf28-33a5-414c-8b03-e8c961251715 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:42.480937 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 32/63] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:42.481804 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c7156-162e-4096-b7a7-728368091b32 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:42.570906 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:42.616521 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fcf28-33a5-414c-8b03-e8c961251715 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:42.625347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c7156-162e-4096-b7a7-728368091b32 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:42.663712 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99df4b-6ba1-43c7-af5b-fdebe1962d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:42.664755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99df4b-6ba1-43c7-af5b-fdebe1962d6f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:42.665186 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99df4b-6ba1-43c7-af5b-fdebe1962d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:42.665899 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99df4b-6ba1-43c7-af5b-fdebe1962d6f None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:42.666995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99df4b-6ba1-43c7-af5b-fdebe1962d6f None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:42.667451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99df4b-6ba1-43c7-af5b-fdebe1962d6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:42.669017 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99df4b-6ba1-43c7-af5b-fdebe1962d6f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:42.670395 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64120a-234e-4276-9eff-dcfa2adf657d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:42.671299 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64120a-234e-4276-9eff-dcfa2adf657d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:42.671735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64120a-234e-4276-9eff-dcfa2adf657d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:42.672144 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64120a-234e-4276-9eff-dcfa2adf657d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:42.678427 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 33/64] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:13:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:42.679536 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99df4b-6ba1-43c7-af5b-fdebe1962d6f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:42.688214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64120a-234e-4276-9eff-dcfa2adf657d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7d64120a-234e-4276-9eff-dcfa2adf657d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:42.694293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:42.695383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:42.696339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:42.698512 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 32/65] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:13:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:42.699475 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64120a-234e-4276-9eff-dcfa2adf657d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:42.776074 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:42.822267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99df4b-6ba1-43c7-af5b-fdebe1962d6f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:42.822779 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64120a-234e-4276-9eff-dcfa2adf657d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:42.857775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef525547-1d2e-4d68-96d7-509f0a41047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:42.858414 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef525547-1d2e-4d68-96d7-509f0a41047f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:42.858699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef525547-1d2e-4d68-96d7-509f0a41047f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:42.858977 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef525547-1d2e-4d68-96d7-509f0a41047f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:42.859192 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402d31a-a56f-485f-8259-782b9d576688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:42.859854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402d31a-a56f-485f-8259-782b9d576688 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:42.860138 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402d31a-a56f-485f-8259-782b9d576688 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:42.860416 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402d31a-a56f-485f-8259-782b9d576688 None admin] PATH_INFO: `/v3/users/a0147318766a4812b77983481386e463` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:42.860889 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:42.870266 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef525547-1d2e-4d68-96d7-509f0a41047f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ef525547-1d2e-4d68-96d7-509f0a41047f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:42.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:42.876482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:13:42.877264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:13:42.877924 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402d31a-a56f-485f-8259-782b9d576688 None admin] RBAC: Authorizing `identity:get_user(user_id=a0147318766a4812b77983481386e463)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:42.878544 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 34/66] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:42.878620 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402d31a-a56f-485f-8259-782b9d576688 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:42.878684 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef525547-1d2e-4d68-96d7-509f0a41047f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:42.879866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 33/67] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:42 2022] GET /identity/v3/users/a0147318766a4812b77983481386e463 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:42.880471 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402d31a-a56f-485f-8259-782b9d576688 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:42.977301 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402d31a-a56f-485f-8259-782b9d576688 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:42.980206 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef525547-1d2e-4d68-96d7-509f0a41047f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:42.993441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:43.011613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6c508-9e75-4199-85dd-df70d4547152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.012242 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6c508-9e75-4199-85dd-df70d4547152 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.012528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6c508-9e75-4199-85dd-df70d4547152 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.012835 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6c508-9e75-4199-85dd-df70d4547152 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.014186 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c09fc-9bc0-4d43-b9ab-061b11fda840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.014799 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c09fc-9bc0-4d43-b9ab-061b11fda840 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.015082 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c09fc-9bc0-4d43-b9ab-061b11fda840 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.015359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c09fc-9bc0-4d43-b9ab-061b11fda840 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.016025 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c09fc-9bc0-4d43-b9ab-061b11fda840 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:43.016327 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c09fc-9bc0-4d43-b9ab-061b11fda840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:43.017760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c09fc-9bc0-4d43-b9ab-061b11fda840 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:43.022888 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb6c508-9e75-4199-85dd-df70d4547152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:43.026059 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 35/68] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:43.027330 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7de7fa-2160-4a93-ba19-f0219e0ba6b3 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.027564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7de7fa-2160-4a93-ba19-f0219e0ba6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ebb6c508-9e75-4199-85dd-df70d4547152 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:43.027761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:43.028602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:13:43.029367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:43.029966 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7de7fa-2160-4a93-ba19-f0219e0ba6b3 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.030030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 34/69] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:13:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:43.030104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 36/70] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:43.030168 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6c508-9e75-4199-85dd-df70d4547152 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.030230 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7de7fa-2160-4a93-ba19-f0219e0ba6b3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.093098 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:43.136821 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6c508-9e75-4199-85dd-df70d4547152 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:43.137628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7de7fa-2160-4a93-ba19-f0219e0ba6b3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:43.170335 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c6223-f74a-4d27-b4d5-79964409b48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.170918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a3d2c-5238-4ae2-8ac2-008f8c7a4212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.171148 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c6223-f74a-4d27-b4d5-79964409b48b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.171437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c6223-f74a-4d27-b4d5-79964409b48b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.171545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a3d2c-5238-4ae2-8ac2-008f8c7a4212 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.171719 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c6223-f74a-4d27-b4d5-79964409b48b None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.171804 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a3d2c-5238-4ae2-8ac2-008f8c7a4212 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.172066 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a3d2c-5238-4ae2-8ac2-008f8c7a4212 None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.172391 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c6223-f74a-4d27-b4d5-79964409b48b None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:43.172730 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c6223-f74a-4d27-b4d5-79964409b48b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:43.173190 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a3d2c-5238-4ae2-8ac2-008f8c7a4212 None admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:43.174156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c6223-f74a-4d27-b4d5-79964409b48b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:43.174599 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a3d2c-5238-4ae2-8ac2-008f8c7a4212 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:43.176210 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 37/71] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:43 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:43.176994 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a3d2c-5238-4ae2-8ac2-008f8c7a4212 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.187953 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 35/72] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:13:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:43.189398 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c6223-f74a-4d27-b4d5-79964409b48b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.230923 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:43.275424 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a3d2c-5238-4ae2-8ac2-008f8c7a4212 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:43.291298 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c6223-f74a-4d27-b4d5-79964409b48b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:43.297573 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:43.308374 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1acb8d-e31c-477c-9324-140fd19bba04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.309052 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1acb8d-e31c-477c-9324-140fd19bba04 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.309337 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1acb8d-e31c-477c-9324-140fd19bba04 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.309616 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1acb8d-e31c-477c-9324-140fd19bba04 None admin] PATH_INFO: `/v3/users/6ac58377d56b4184a21764af3c195e68` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.310765 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1acb8d-e31c-477c-9324-140fd19bba04 None admin] RBAC: Authorizing `identity:get_user(user_id=6ac58377d56b4184a21764af3c195e68)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:43.312217 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1acb8d-e31c-477c-9324-140fd19bba04 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:43.313833 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 38/73] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:43 2022] GET /identity/v3/users/6ac58377d56b4184a21764af3c195e68 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:43.317698 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b75bc-a80a-44a2-8edb-0f5594f99e49 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.317896 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b75bc-a80a-44a2-8edb-0f5594f99e49 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.318177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b75bc-a80a-44a2-8edb-0f5594f99e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.326607 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834105f-98bb-45df-8225-2d1e5ec252f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.327222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834105f-98bb-45df-8225-2d1e5ec252f8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.327501 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834105f-98bb-45df-8225-2d1e5ec252f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.327781 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834105f-98bb-45df-8225-2d1e5ec252f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.328438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834105f-98bb-45df-8225-2d1e5ec252f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:43.328843 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834105f-98bb-45df-8225-2d1e5ec252f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:43.330208 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834105f-98bb-45df-8225-2d1e5ec252f8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:43.335800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-506b75bc-a80a-44a2-8edb-0f5594f99e49 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:43.337039 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-506b75bc-a80a-44a2-8edb-0f5594f99e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:43.338449 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 36/74] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:13:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:43.339212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834105f-98bb-45df-8225-2d1e5ec252f8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.406584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 39/75] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:43.407418 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b75bc-a80a-44a2-8edb-0f5594f99e49 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.437585 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:43.441215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834105f-98bb-45df-8225-2d1e5ec252f8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:43.479508 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720257a2-1e87-4a55-afd4-7b245fad5f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.480269 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720257a2-1e87-4a55-afd4-7b245fad5f3f None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.480578 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720257a2-1e87-4a55-afd4-7b245fad5f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.481119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720257a2-1e87-4a55-afd4-7b245fad5f3f None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47/users/a0147318766a4812b77983481386e463/roles/6914eb62761341d7b8640a878225f115` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.482908 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720257a2-1e87-4a55-afd4-7b245fad5f3f None admin] RBAC: Authorizing `identity:create_grant(project_id=77d8e654d0884c519a11a6471e3aee47, user_id=a0147318766a4812b77983481386e463, role_id=6914eb62761341d7b8640a878225f115)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:43.484339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720257a2-1e87-4a55-afd4-7b245fad5f3f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:43.508943 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 37/76] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:13:43 2022] PUT /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47/users/a0147318766a4812b77983481386e463/roles/6914eb62761341d7b8640a878225f115 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:13:43.510074 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720257a2-1e87-4a55-afd4-7b245fad5f3f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.527097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b75bc-a80a-44a2-8edb-0f5594f99e49 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:43.565104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:43.573345 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c0ccc-3b9e-4e24-bd0d-63f3a20ac9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.574946 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c0ccc-3b9e-4e24-bd0d-63f3a20ac9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.575444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c0ccc-3b9e-4e24-bd0d-63f3a20ac9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.575813 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c0ccc-3b9e-4e24-bd0d-63f3a20ac9f4 None admin] PATH_INFO: `/v3/users/a0147318766a4812b77983481386e463` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.577517 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c0ccc-3b9e-4e24-bd0d-63f3a20ac9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=a0147318766a4812b77983481386e463)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:43.580111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c0ccc-3b9e-4e24-bd0d-63f3a20ac9f4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:43.582709 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 40/77] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:43 2022] GET /identity/v3/users/a0147318766a4812b77983481386e463 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:43.589176 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c0ccc-3b9e-4e24-bd0d-63f3a20ac9f4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.651236 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720257a2-1e87-4a55-afd4-7b245fad5f3f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:43.699227 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6dd5b-fe0f-4384-bac2-bb360f94a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.699421 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6dd5b-fe0f-4384-bac2-bb360f94a80e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.699636 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6dd5b-fe0f-4384-bac2-bb360f94a80e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.699997 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6dd5b-fe0f-4384-bac2-bb360f94a80e None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.701825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6dd5b-fe0f-4384-bac2-bb360f94a80e None admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:43.703140 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6dd5b-fe0f-4384-bac2-bb360f94a80e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:43.704640 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 38/78] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:43 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:43.705498 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6dd5b-fe0f-4384-bac2-bb360f94a80e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.723530 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c0ccc-3b9e-4e24-bd0d-63f3a20ac9f4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:43.738644 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:43.762518 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b069757-e6c0-4c51-9b47-c2239b336916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.763161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b069757-e6c0-4c51-9b47-c2239b336916 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.763448 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b069757-e6c0-4c51-9b47-c2239b336916 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.763717 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b069757-e6c0-4c51-9b47-c2239b336916 None admin] PATH_INFO: `/v3/users/a0147318766a4812b77983481386e463` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.764903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b069757-e6c0-4c51-9b47-c2239b336916 None admin] RBAC: Authorizing `identity:get_user(user_id=a0147318766a4812b77983481386e463)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:43.766345 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b069757-e6c0-4c51-9b47-c2239b336916 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:43.768283 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 41/79] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:43 2022] GET /identity/v3/users/a0147318766a4812b77983481386e463 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:43.769580 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b069757-e6c0-4c51-9b47-c2239b336916 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.824529 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6dd5b-fe0f-4384-bac2-bb360f94a80e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:43.829032 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:43.861787 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb051db4-6f1e-4b3f-a6b7-9bd0fe8add17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.862496 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb051db4-6f1e-4b3f-a6b7-9bd0fe8add17 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.862740 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb051db4-6f1e-4b3f-a6b7-9bd0fe8add17 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.863019 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb051db4-6f1e-4b3f-a6b7-9bd0fe8add17 None admin] PATH_INFO: `/v3/projects/260dfcd27fc6476da82f72872627a534` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.864177 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb051db4-6f1e-4b3f-a6b7-9bd0fe8add17 None admin] RBAC: Authorizing `identity:get_project(project_id=260dfcd27fc6476da82f72872627a534)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:43.865660 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb051db4-6f1e-4b3f-a6b7-9bd0fe8add17 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:43.873224 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 39/80] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:43 2022] GET /identity/v3/projects/260dfcd27fc6476da82f72872627a534 => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:43.873224 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb051db4-6f1e-4b3f-a6b7-9bd0fe8add17 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.881525 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b069757-e6c0-4c51-9b47-c2239b336916 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:43.917283 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514431b-a0fb-402f-b13c-4e47bde32262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:43.917924 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514431b-a0fb-402f-b13c-4e47bde32262 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:43.918210 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514431b-a0fb-402f-b13c-4e47bde32262 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:43.918485 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514431b-a0fb-402f-b13c-4e47bde32262 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:43.919609 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514431b-a0fb-402f-b13c-4e47bde32262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:43.921092 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514431b-a0fb-402f-b13c-4e47bde32262 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:43.922665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 42/81] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:43.924732 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514431b-a0fb-402f-b13c-4e47bde32262 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:43.976062 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:43.983424 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb051db4-6f1e-4b3f-a6b7-9bd0fe8add17 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:44.021751 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd3a0c-71ac-4efe-8a48-e78a80ea818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:44.022426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd3a0c-71ac-4efe-8a48-e78a80ea818d None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:44.022713 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd3a0c-71ac-4efe-8a48-e78a80ea818d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:44.022991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd3a0c-71ac-4efe-8a48-e78a80ea818d None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47/users/6ac58377d56b4184a21764af3c195e68/roles/81d287a8b606458d87c9607e8b48ac58` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:44.024999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd3a0c-71ac-4efe-8a48-e78a80ea818d None admin] RBAC: Authorizing `identity:create_grant(project_id=77d8e654d0884c519a11a6471e3aee47, user_id=6ac58377d56b4184a21764af3c195e68, role_id=81d287a8b606458d87c9607e8b48ac58)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:44.026463 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd3a0c-71ac-4efe-8a48-e78a80ea818d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:44.032274 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514431b-a0fb-402f-b13c-4e47bde32262 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:44.040226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 40/82] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:13:43 2022] PUT /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47/users/6ac58377d56b4184a21764af3c195e68/roles/81d287a8b606458d87c9607e8b48ac58 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:13:44.047050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd3a0c-71ac-4efe-8a48-e78a80ea818d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:44.064353 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:44.075705 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311ac84-99e0-406e-a9f0-d30d0c155243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:44.076819 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ac84-99e0-406e-a9f0-d30d0c155243 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:44.077424 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ac84-99e0-406e-a9f0-d30d0c155243 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:44.077921 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ac84-99e0-406e-a9f0-d30d0c155243 None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:44.079444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311ac84-99e0-406e-a9f0-d30d0c155243 None admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:44.082055 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311ac84-99e0-406e-a9f0-d30d0c155243 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:44.084103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 43/83] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:43 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:44.085327 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311ac84-99e0-406e-a9f0-d30d0c155243 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:44.192359 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd3a0c-71ac-4efe-8a48-e78a80ea818d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:44.215978 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311ac84-99e0-406e-a9f0-d30d0c155243 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:44.247584 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc0ccc-08c0-4578-8bc9-95f36ca72b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:44.248267 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc0ccc-08c0-4578-8bc9-95f36ca72b53 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:44.248614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc0ccc-08c0-4578-8bc9-95f36ca72b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:44.248870 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc0ccc-08c0-4578-8bc9-95f36ca72b53 None admin] PATH_INFO: `/v3/projects/260dfcd27fc6476da82f72872627a534/users/a0147318766a4812b77983481386e463/roles/6914eb62761341d7b8640a878225f115` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:44.251564 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc0ccc-08c0-4578-8bc9-95f36ca72b53 None admin] RBAC: Authorizing `identity:create_grant(project_id=260dfcd27fc6476da82f72872627a534, user_id=a0147318766a4812b77983481386e463, role_id=6914eb62761341d7b8640a878225f115)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:44.253104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc0ccc-08c0-4578-8bc9-95f36ca72b53 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:44.261082 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9bda2-346e-4d38-87f7-20bf66b7cd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:44.262057 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9bda2-346e-4d38-87f7-20bf66b7cd43 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:44.262619 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9bda2-346e-4d38-87f7-20bf66b7cd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:44.263084 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9bda2-346e-4d38-87f7-20bf66b7cd43 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:44.264083 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9bda2-346e-4d38-87f7-20bf66b7cd43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:44.266089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9bda2-346e-4d38-87f7-20bf66b7cd43 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:44.267570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 41/84] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:13:44 2022] PUT /identity/v3/projects/260dfcd27fc6476da82f72872627a534/users/a0147318766a4812b77983481386e463/roles/6914eb62761341d7b8640a878225f115 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:13:44.268991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc0ccc-08c0-4578-8bc9-95f36ca72b53 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:44.290237 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:44.305533 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 44/85] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:13:44 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:13:44.390273 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc0ccc-08c0-4578-8bc9-95f36ca72b53 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:44.431505 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda3bc8-0908-4196-82aa-e49a26fba6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:44.432196 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda3bc8-0908-4196-82aa-e49a26fba6be None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:44.432498 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda3bc8-0908-4196-82aa-e49a26fba6be None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:44.432795 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda3bc8-0908-4196-82aa-e49a26fba6be None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47/users/a0147318766a4812b77983481386e463/roles/1255151437d94364acba841f4215321c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:44.435045 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda3bc8-0908-4196-82aa-e49a26fba6be None admin] RBAC: Authorizing `identity:create_grant(project_id=77d8e654d0884c519a11a6471e3aee47, user_id=a0147318766a4812b77983481386e463, role_id=1255151437d94364acba841f4215321c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:44.436520 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda3bc8-0908-4196-82aa-e49a26fba6be None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:44.458801 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 42/86] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:13:44 2022] PUT /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47/users/a0147318766a4812b77983481386e463/roles/1255151437d94364acba841f4215321c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:13:46.134844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da41a6f-2cb3-4ade-ac05-e6e4c91e3208 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:46.135328 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da41a6f-2cb3-4ade-ac05-e6e4c91e3208 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:46.135402 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da41a6f-2cb3-4ade-ac05-e6e4c91e3208 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:46.136775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 45/87] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:46.142293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483a9e8-a2b3-4bb2-9e4d-d0faa2fe35f7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:46.143127 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483a9e8-a2b3-4bb2-9e4d-d0faa2fe35f7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:46.143682 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483a9e8-a2b3-4bb2-9e4d-d0faa2fe35f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:46.185579 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b483a9e8-a2b3-4bb2-9e4d-d0faa2fe35f7 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:46.187144 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b483a9e8-a2b3-4bb2-9e4d-d0faa2fe35f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:46.286278 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 43/88] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:13:46.791194 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85235604-427c-42e6-af0d-e4787b0a30ea None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:46.791557 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85235604-427c-42e6-af0d-e4787b0a30ea None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:46.791760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85235604-427c-42e6-af0d-e4787b0a30ea None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:46.793142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 46/89] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:46.798317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28bcd-3556-4a6f-aef4-e7e8ab4677dd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:46.798840 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28bcd-3556-4a6f-aef4-e7e8ab4677dd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:46.799340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28bcd-3556-4a6f-aef4-e7e8ab4677dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:46.818538 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:46.830087 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f0e28bcd-3556-4a6f-aef4-e7e8ab4677dd None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:46.831569 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f0e28bcd-3556-4a6f-aef4-e7e8ab4677dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:46.844910 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85235604-427c-42e6-af0d-e4787b0a30ea None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:46.921540 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:46.966173 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 44/90] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:47.025857 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85235604-427c-42e6-af0d-e4787b0a30ea None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:47.085487 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc97373-a221-4abe-8dbb-71e9fe71829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:47.086495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc97373-a221-4abe-8dbb-71e9fe71829f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:47.086786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc97373-a221-4abe-8dbb-71e9fe71829f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:47.087066 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc97373-a221-4abe-8dbb-71e9fe71829f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:47.109190 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc97373-a221-4abe-8dbb-71e9fe71829f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4cc97373-a221-4abe-8dbb-71e9fe71829f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:47.114773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:47.115610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:13:47.116355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 47/91] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:47.121880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e28bcd-3556-4a6f-aef4-e7e8ab4677dd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:47.266758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a25d4-ef77-4eda-b257-8eaed3778726 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:47.267236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a25d4-ef77-4eda-b257-8eaed3778726 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:47.267236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a25d4-ef77-4eda-b257-8eaed3778726 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:47.270281 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 48/92] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:47.275742 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f58e0-b0d0-4550-b09d-1712646a2cfc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:47.276035 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f58e0-b0d0-4550-b09d-1712646a2cfc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:47.276316 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f58e0-b0d0-4550-b09d-1712646a2cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:47.290733 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e28bcd-3556-4a6f-aef4-e7e8ab4677dd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:47.308795 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-957f58e0-b0d0-4550-b09d-1712646a2cfc None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:47.310016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-957f58e0-b0d0-4550-b09d-1712646a2cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:47.358984 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e3206-70a8-4b64-a983-d890ba3862bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:47.359906 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e3206-70a8-4b64-a983-d890ba3862bb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:47.360199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e3206-70a8-4b64-a983-d890ba3862bb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:47.360478 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e3206-70a8-4b64-a983-d890ba3862bb None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:47.361186 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e3206-70a8-4b64-a983-d890ba3862bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:47.361494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e3206-70a8-4b64-a983-d890ba3862bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:47.365475 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e3206-70a8-4b64-a983-d890ba3862bb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:47.371177 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:47.386251 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 45/93] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:13:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:47.392164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e3206-70a8-4b64-a983-d890ba3862bb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:47.466764 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 49/94] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:47.467702 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f58e0-b0d0-4550-b09d-1712646a2cfc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:47.547459 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:47.571629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e3206-70a8-4b64-a983-d890ba3862bb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:47.614033 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f58e0-b0d0-4550-b09d-1712646a2cfc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:47.614574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d0365-0bbd-418e-8606-425b39388e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:47.615227 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d0365-0bbd-418e-8606-425b39388e15 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:47.615783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d0365-0bbd-418e-8606-425b39388e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:47.616067 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d0365-0bbd-418e-8606-425b39388e15 None admin] PATH_INFO: `/v3/users/a0147318766a4812b77983481386e463` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:47.618863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d0365-0bbd-418e-8606-425b39388e15 None admin] RBAC: Authorizing `identity:get_user(user_id=a0147318766a4812b77983481386e463)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:47.620326 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d0365-0bbd-418e-8606-425b39388e15 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:47.625991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 46/95] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:47 2022] GET /identity/v3/users/a0147318766a4812b77983481386e463 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:47.627288 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8756501-1839-49af-9b81-30d1cdcef132 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:47.627416 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8756501-1839-49af-9b81-30d1cdcef132 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:47.628435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8756501-1839-49af-9b81-30d1cdcef132 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:47.632890 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 47/96] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:47.633632 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8756501-1839-49af-9b81-30d1cdcef132 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:47.696918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f220d-aa40-46bb-94b7-9e53f9e0fdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:47.697656 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f220d-aa40-46bb-94b7-9e53f9e0fdeb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:47.697949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f220d-aa40-46bb-94b7-9e53f9e0fdeb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:47.698219 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f220d-aa40-46bb-94b7-9e53f9e0fdeb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:47.714175 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:47.726845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f220d-aa40-46bb-94b7-9e53f9e0fdeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-716f220d-aa40-46bb-94b7-9e53f9e0fdeb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:47.731864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:47.732758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:13:47.733916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 50/97] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:47.741945 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe39628-adf1-4e53-8385-b991a97eeb91 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:47.742373 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe39628-adf1-4e53-8385-b991a97eeb91 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:47.742830 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe39628-adf1-4e53-8385-b991a97eeb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:47.761035 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8756501-1839-49af-9b81-30d1cdcef132 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:47.784882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0fe39628-adf1-4e53-8385-b991a97eeb91 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:47.786087 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0fe39628-adf1-4e53-8385-b991a97eeb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:47.798886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efe063-f78c-43a9-b299-024bd1fdd9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:47.799807 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efe063-f78c-43a9-b299-024bd1fdd9ab None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:47.800328 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efe063-f78c-43a9-b299-024bd1fdd9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:47.800841 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efe063-f78c-43a9-b299-024bd1fdd9ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:47.875600 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efe063-f78c-43a9-b299-024bd1fdd9ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:47.877437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efe063-f78c-43a9-b299-024bd1fdd9ab None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:47.920436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 51/98] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:47.920436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe39628-adf1-4e53-8385-b991a97eeb91 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:47.957174 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:47.990029 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:48.053178 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe39628-adf1-4e53-8385-b991a97eeb91 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:48.053624 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 48/99] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:13:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:13:48.055584 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efe063-f78c-43a9-b299-024bd1fdd9ab None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:48.089233 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ac04b-e817-400b-8bba-ee9854a725af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:48.090111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ac04b-e817-400b-8bba-ee9854a725af None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.090483 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ac04b-e817-400b-8bba-ee9854a725af None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.090868 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ac04b-e817-400b-8bba-ee9854a725af None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.091671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ac04b-e817-400b-8bba-ee9854a725af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:48.092099 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ac04b-e817-400b-8bba-ee9854a725af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:48.093924 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ac04b-e817-400b-8bba-ee9854a725af None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:48.103350 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 52/100] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:48.104932 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc31d63-283a-4ef1-87ff-013cdf790e6a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.105550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc31d63-283a-4ef1-87ff-013cdf790e6a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.105759 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc31d63-283a-4ef1-87ff-013cdf790e6a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.107087 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 53/101] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:48.112059 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc31d63-283a-4ef1-87ff-013cdf790e6a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:48.169423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efe063-f78c-43a9-b299-024bd1fdd9ab None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:48.206570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:48.208681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4a65e-92da-4ed7-a096-49f1b539c70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:48.209326 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4a65e-92da-4ed7-a096-49f1b539c70d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.209609 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4a65e-92da-4ed7-a096-49f1b539c70d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.209883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4a65e-92da-4ed7-a096-49f1b539c70d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.218088 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc31d63-283a-4ef1-87ff-013cdf790e6a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:48.223008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4a65e-92da-4ed7-a096-49f1b539c70d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c1f4a65e-92da-4ed7-a096-49f1b539c70d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:48.229548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:48.230423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:13:48.231169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:48.231753 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 49/102] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:13:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:48.232673 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4a65e-92da-4ed7-a096-49f1b539c70d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:48.258355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f6562-44fa-4229-b1c4-0ee2a1244b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:48.258997 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6562-44fa-4229-b1c4-0ee2a1244b88 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.259282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6562-44fa-4229-b1c4-0ee2a1244b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.259559 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6562-44fa-4229-b1c4-0ee2a1244b88 None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.260712 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f6562-44fa-4229-b1c4-0ee2a1244b88 None admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:48.262125 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:48.262245 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f6562-44fa-4229-b1c4-0ee2a1244b88 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:48.263836 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 54/103] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:48 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:48.265422 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ed4ff-ffb6-47d6-8b4a-dee8bca05b1b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.265712 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ed4ff-ffb6-47d6-8b4a-dee8bca05b1b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.266012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ed4ff-ffb6-47d6-8b4a-dee8bca05b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.284510 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-260ed4ff-ffb6-47d6-8b4a-dee8bca05b1b None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:48.285872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-260ed4ff-ffb6-47d6-8b4a-dee8bca05b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:48.338528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4a65e-92da-4ed7-a096-49f1b539c70d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:48.350979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 55/104] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:48.360666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902bb3-87e0-4e71-bf00-c7427719b60e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.360666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902bb3-87e0-4e71-bf00-c7427719b60e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.360666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902bb3-87e0-4e71-bf00-c7427719b60e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.360666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 56/105] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:48.360666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902bb3-87e0-4e71-bf00-c7427719b60e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:48.383823 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe7a5f-92db-40d5-bdb9-ef42dcd3a2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:48.384454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe7a5f-92db-40d5-bdb9-ef42dcd3a2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.384834 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe7a5f-92db-40d5-bdb9-ef42dcd3a2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.385108 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe7a5f-92db-40d5-bdb9-ef42dcd3a2a5 None admin] PATH_INFO: `/v3/users/6ac58377d56b4184a21764af3c195e68` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.386228 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe7a5f-92db-40d5-bdb9-ef42dcd3a2a5 None admin] RBAC: Authorizing `identity:get_user(user_id=6ac58377d56b4184a21764af3c195e68)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:48.387671 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe7a5f-92db-40d5-bdb9-ef42dcd3a2a5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:48.389270 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 50/106] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:48 2022] GET /identity/v3/users/6ac58377d56b4184a21764af3c195e68 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:48.390033 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe7a5f-92db-40d5-bdb9-ef42dcd3a2a5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:48.458432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:48.471498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902bb3-87e0-4e71-bf00-c7427719b60e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:48.493001 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:48.499623 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe7a5f-92db-40d5-bdb9-ef42dcd3a2a5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:48.510371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09b65f-66d9-42cf-8c84-88c1ea4c8222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:48.510982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09b65f-66d9-42cf-8c84-88c1ea4c8222 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.511261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09b65f-66d9-42cf-8c84-88c1ea4c8222 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.511531 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09b65f-66d9-42cf-8c84-88c1ea4c8222 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.512189 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09b65f-66d9-42cf-8c84-88c1ea4c8222 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:48.512489 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09b65f-66d9-42cf-8c84-88c1ea4c8222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:48.513916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09b65f-66d9-42cf-8c84-88c1ea4c8222 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:48.523125 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 57/107] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:13:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:48.528237 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09b65f-66d9-42cf-8c84-88c1ea4c8222 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:48.539018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad827cc-e8cd-4d30-af1f-6b1e45385ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:48.539825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad827cc-e8cd-4d30-af1f-6b1e45385ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.540248 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad827cc-e8cd-4d30-af1f-6b1e45385ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.540547 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad827cc-e8cd-4d30-af1f-6b1e45385ba9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.554376 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad827cc-e8cd-4d30-af1f-6b1e45385ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2ad827cc-e8cd-4d30-af1f-6b1e45385ba9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:48.559259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:48.560083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:48.560862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 51/108] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:48.561813 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aceeb-e6bc-45c7-b513-826c450a69af None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.561813 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aceeb-e6bc-45c7-b513-826c450a69af None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.562092 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aceeb-e6bc-45c7-b513-826c450a69af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.581392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d18aceeb-e6bc-45c7-b513-826c450a69af None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:48.582598 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d18aceeb-e6bc-45c7-b513-826c450a69af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:48.638182 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09b65f-66d9-42cf-8c84-88c1ea4c8222 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:48.651760 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 52/109] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:48.652669 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18aceeb-e6bc-45c7-b513-826c450a69af None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:48.673459 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412f345-55d3-404e-8c5e-1be284d1cfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:48.674095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412f345-55d3-404e-8c5e-1be284d1cfbf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.674369 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412f345-55d3-404e-8c5e-1be284d1cfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.674647 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412f345-55d3-404e-8c5e-1be284d1cfbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.675919 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412f345-55d3-404e-8c5e-1be284d1cfbf None admin] RBAC: Adding query filter params (role.id=6914eb62761341d7b8640a878225f115, scope.project.id=77d8e654d0884c519a11a6471e3aee47, user.id=a0147318766a4812b77983481386e463) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:48.676235 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412f345-55d3-404e-8c5e-1be284d1cfbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:48.677731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412f345-55d3-404e-8c5e-1be284d1cfbf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:48.685157 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 58/110] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:13:48 2022] GET /identity/v3/role_assignments?user.id=a0147318766a4812b77983481386e463&scope.project.id=77d8e654d0884c519a11a6471e3aee47&role.id=6914eb62761341d7b8640a878225f115 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:48.685775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412f345-55d3-404e-8c5e-1be284d1cfbf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:48.705305 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:48.766267 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:48.790283 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18aceeb-e6bc-45c7-b513-826c450a69af None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:48.831641 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208a043-847a-48a6-9bc4-2f5f79055067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:48.832276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a043-847a-48a6-9bc4-2f5f79055067 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.832595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a043-847a-48a6-9bc4-2f5f79055067 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.832882 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a043-847a-48a6-9bc4-2f5f79055067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.839144 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412f345-55d3-404e-8c5e-1be284d1cfbf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:48.889383 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a66e1-c045-46b5-8608-51652198d1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:48.890346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a66e1-c045-46b5-8608-51652198d1eb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:48.890859 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a66e1-c045-46b5-8608-51652198d1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:48.891329 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a66e1-c045-46b5-8608-51652198d1eb None admin] PATH_INFO: `/v3/users/a0147318766a4812b77983481386e463` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:48.892971 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a66e1-c045-46b5-8608-51652198d1eb None admin] RBAC: Authorizing `identity:get_user(user_id=a0147318766a4812b77983481386e463)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:48.895618 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a66e1-c045-46b5-8608-51652198d1eb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:48.897539 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 59/111] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:48 2022] GET /identity/v3/users/a0147318766a4812b77983481386e463 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:48.899071 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a66e1-c045-46b5-8608-51652198d1eb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:48.916613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208a043-847a-48a6-9bc4-2f5f79055067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:48.919025 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208a043-847a-48a6-9bc4-2f5f79055067 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:49.006083 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:49.006508 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:49.022100 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a66e1-c045-46b5-8608-51652198d1eb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:49.023679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 53/112] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:13:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:13:49.024501 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208a043-847a-48a6-9bc4-2f5f79055067 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:49.055838 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc885515-3a68-4826-b43e-24b9d33ea8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:49.056455 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc885515-3a68-4826-b43e-24b9d33ea8da None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:49.056778 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc885515-3a68-4826-b43e-24b9d33ea8da None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:49.057056 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc885515-3a68-4826-b43e-24b9d33ea8da None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:49.057721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc885515-3a68-4826-b43e-24b9d33ea8da None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:49.058030 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc885515-3a68-4826-b43e-24b9d33ea8da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:49.059425 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc885515-3a68-4826-b43e-24b9d33ea8da None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:49.067977 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 60/113] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:13:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:49.068739 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc885515-3a68-4826-b43e-24b9d33ea8da None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:49.123760 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208a043-847a-48a6-9bc4-2f5f79055067 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:49.158385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea2e5e-105b-4d1d-a469-f9eda0bf4ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:49.158954 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea2e5e-105b-4d1d-a469-f9eda0bf4ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:49.159734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea2e5e-105b-4d1d-a469-f9eda0bf4ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:49.160015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea2e5e-105b-4d1d-a469-f9eda0bf4ac7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:49.161153 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea2e5e-105b-4d1d-a469-f9eda0bf4ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:49.162591 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea2e5e-105b-4d1d-a469-f9eda0bf4ac7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:49.164167 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 54/114] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:49.164952 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea2e5e-105b-4d1d-a469-f9eda0bf4ac7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:49.166332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc885515-3a68-4826-b43e-24b9d33ea8da None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:49.201120 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd43e3c-fe79-41d4-8d40-09b6ad8d764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:49.201732 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43e3c-fe79-41d4-8d40-09b6ad8d764c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:49.202017 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43e3c-fe79-41d4-8d40-09b6ad8d764c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:49.202388 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43e3c-fe79-41d4-8d40-09b6ad8d764c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:49.217822 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:49.218446 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:49.264552 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea2e5e-105b-4d1d-a469-f9eda0bf4ac7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:49.270917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd43e3c-fe79-41d4-8d40-09b6ad8d764c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:49.272436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd43e3c-fe79-41d4-8d40-09b6ad8d764c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:49.297549 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3f732-ece0-4ebd-b122-266ff4353ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:49.298174 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3f732-ece0-4ebd-b122-266ff4353ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:49.298451 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3f732-ece0-4ebd-b122-266ff4353ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:49.298729 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3f732-ece0-4ebd-b122-266ff4353ab7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:49.308506 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3f732-ece0-4ebd-b122-266ff4353ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1bd3f732-ece0-4ebd-b122-266ff4353ab7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:49.313312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:49.314170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:49.314940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 55/115] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:49.316092 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3f732-ece0-4ebd-b122-266ff4353ab7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:49.375393 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 61/116] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:13:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:13:49.375393 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd43e3c-fe79-41d4-8d40-09b6ad8d764c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:49.416476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:49.417515 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3f732-ece0-4ebd-b122-266ff4353ab7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:49.431762 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:49.453603 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d268-2442-4376-959b-6c88abc80282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:49.454228 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d268-2442-4376-959b-6c88abc80282 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:49.454511 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d268-2442-4376-959b-6c88abc80282 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:49.454779 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d268-2442-4376-959b-6c88abc80282 None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:49.455894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850d268-2442-4376-959b-6c88abc80282 None admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:49.457397 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850d268-2442-4376-959b-6c88abc80282 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:49.459050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 56/117] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:49 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:49.464986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d268-2442-4376-959b-6c88abc80282 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:49.468778 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd43e3c-fe79-41d4-8d40-09b6ad8d764c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:49.501571 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667fc4b-7aad-4885-b600-50901825cbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:49.502213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667fc4b-7aad-4885-b600-50901825cbe6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:49.502486 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667fc4b-7aad-4885-b600-50901825cbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:49.502761 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667fc4b-7aad-4885-b600-50901825cbe6 None admin] PATH_INFO: `/v3/users/a0147318766a4812b77983481386e463` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:49.503907 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667fc4b-7aad-4885-b600-50901825cbe6 None admin] RBAC: Authorizing `identity:get_user(user_id=a0147318766a4812b77983481386e463)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:49.505389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667fc4b-7aad-4885-b600-50901825cbe6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:49.506946 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 62/118] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:49 2022] GET /identity/v3/users/a0147318766a4812b77983481386e463 => generated 275 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:49.507705 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667fc4b-7aad-4885-b600-50901825cbe6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:49.563328 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d268-2442-4376-959b-6c88abc80282 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:49.597165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fc838-48ec-4a38-acb0-f80bb8b447bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:49.597772 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fc838-48ec-4a38-acb0-f80bb8b447bf None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:49.598071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fc838-48ec-4a38-acb0-f80bb8b447bf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:49.598345 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fc838-48ec-4a38-acb0-f80bb8b447bf None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:49.598988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fc838-48ec-4a38-acb0-f80bb8b447bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:49.600437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fc838-48ec-4a38-acb0-f80bb8b447bf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:49.603638 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667fc4b-7aad-4885-b600-50901825cbe6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:49.617877 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:49.626584 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 57/119] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:13:49 2022] POST /identity/v3/projects => generated 302 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:13:49.626584 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fc838-48ec-4a38-acb0-f80bb8b447bf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:49.645943 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f9898-af01-4bae-bd86-abc0b23f6c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:49.646925 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f9898-af01-4bae-bd86-abc0b23f6c77 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:49.647232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f9898-af01-4bae-bd86-abc0b23f6c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:49.647518 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f9898-af01-4bae-bd86-abc0b23f6c77 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:49.648223 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f9898-af01-4bae-bd86-abc0b23f6c77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:49.648553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f9898-af01-4bae-bd86-abc0b23f6c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:49.650043 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f9898-af01-4bae-bd86-abc0b23f6c77 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:49.659444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 63/120] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:13:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:49.662349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f9898-af01-4bae-bd86-abc0b23f6c77 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:49.669434 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:49.755368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fc838-48ec-4a38-acb0-f80bb8b447bf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:49.777731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f9898-af01-4bae-bd86-abc0b23f6c77 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:49.793981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5457749-0a4a-47a8-9303-66ac3fa53637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:49.794639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5457749-0a4a-47a8-9303-66ac3fa53637 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:49.794923 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5457749-0a4a-47a8-9303-66ac3fa53637 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:49.795198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5457749-0a4a-47a8-9303-66ac3fa53637 None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:49.796372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5457749-0a4a-47a8-9303-66ac3fa53637 None admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:49.797954 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5457749-0a4a-47a8-9303-66ac3fa53637 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:49.804706 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 58/121] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:49 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:49.804706 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5457749-0a4a-47a8-9303-66ac3fa53637 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:49.822680 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93fe14-966f-4e1b-a022-95afc72bd7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:49.823299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93fe14-966f-4e1b-a022-95afc72bd7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:49.823584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93fe14-966f-4e1b-a022-95afc72bd7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:49.823859 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93fe14-966f-4e1b-a022-95afc72bd7a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:49.824760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93fe14-966f-4e1b-a022-95afc72bd7a7 None admin] RBAC: Adding query filter params (role.id=81d287a8b606458d87c9607e8b48ac58, scope.project.id=77d8e654d0884c519a11a6471e3aee47, user.id=6ac58377d56b4184a21764af3c195e68) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:49.825070 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93fe14-966f-4e1b-a022-95afc72bd7a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:49.826516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93fe14-966f-4e1b-a022-95afc72bd7a7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:49.833903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 64/122] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:13:49 2022] GET /identity/v3/role_assignments?user.id=6ac58377d56b4184a21764af3c195e68&scope.project.id=77d8e654d0884c519a11a6471e3aee47&role.id=81d287a8b606458d87c9607e8b48ac58 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:49.841548 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93fe14-966f-4e1b-a022-95afc72bd7a7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:49.906161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:49.927527 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:49.930924 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5457749-0a4a-47a8-9303-66ac3fa53637 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:49.974238 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c5ee9-7b93-4d2c-b71e-950d64417046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:49.975188 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c5ee9-7b93-4d2c-b71e-950d64417046 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:49.975614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c5ee9-7b93-4d2c-b71e-950d64417046 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:49.975995 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c5ee9-7b93-4d2c-b71e-950d64417046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:49.995977 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93fe14-966f-4e1b-a022-95afc72bd7a7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:50.041662 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba01ced-6ed8-4094-9654-4ac9ce66f3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:50.042460 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba01ced-6ed8-4094-9654-4ac9ce66f3ec None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:50.042744 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba01ced-6ed8-4094-9654-4ac9ce66f3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:50.043019 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba01ced-6ed8-4094-9654-4ac9ce66f3ec None admin] PATH_INFO: `/v3/users/40ad1f973cfa4a84aa61dd53e8afe1f6` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:50.059425 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba01ced-6ed8-4094-9654-4ac9ce66f3ec None admin] RBAC: Authorizing `identity:get_user(user_id=40ad1f973cfa4a84aa61dd53e8afe1f6)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:50.060971 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba01ced-6ed8-4094-9654-4ac9ce66f3ec None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:50.062822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 65/123] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:49 2022] GET /identity/v3/users/40ad1f973cfa4a84aa61dd53e8afe1f6 => generated 289 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:50.063510 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba01ced-6ed8-4094-9654-4ac9ce66f3ec None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:50.064263 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c5ee9-7b93-4d2c-b71e-950d64417046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:50.066567 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c5ee9-7b93-4d2c-b71e-950d64417046 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:50.172421 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:50.177414 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba01ced-6ed8-4094-9654-4ac9ce66f3ec None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:50.178059 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:50.181723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 59/124] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:13:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:13:50.186932 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c5ee9-7b93-4d2c-b71e-950d64417046 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:50.216837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c055a2-2422-44c7-ac2b-234392cd0192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:50.217447 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c055a2-2422-44c7-ac2b-234392cd0192 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:50.217726 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c055a2-2422-44c7-ac2b-234392cd0192 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:50.217997 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c055a2-2422-44c7-ac2b-234392cd0192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:50.218721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c055a2-2422-44c7-ac2b-234392cd0192 None admin] RBAC: Adding query filter params (role.id=1255151437d94364acba841f4215321c, scope.project.id=77d8e654d0884c519a11a6471e3aee47, user.id=a0147318766a4812b77983481386e463) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:50.219025 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c055a2-2422-44c7-ac2b-234392cd0192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:50.220473 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c055a2-2422-44c7-ac2b-234392cd0192 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:50.226991 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 66/125] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:13:50 2022] GET /identity/v3/role_assignments?user.id=a0147318766a4812b77983481386e463&scope.project.id=77d8e654d0884c519a11a6471e3aee47&role.id=1255151437d94364acba841f4215321c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:50.227791 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c055a2-2422-44c7-ac2b-234392cd0192 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:50.305863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c5ee9-7b93-4d2c-b71e-950d64417046 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:50.350484 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8766f-d8ce-4598-806a-a3bcc70a6dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:50.351560 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8766f-d8ce-4598-806a-a3bcc70a6dfd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:50.352057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8766f-d8ce-4598-806a-a3bcc70a6dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:50.352569 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8766f-d8ce-4598-806a-a3bcc70a6dfd None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:50.354060 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8766f-d8ce-4598-806a-a3bcc70a6dfd None admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:50.354410 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c055a2-2422-44c7-ac2b-234392cd0192 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:50.356122 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8766f-d8ce-4598-806a-a3bcc70a6dfd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:50.358259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 60/126] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:50 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:50.365649 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8766f-d8ce-4598-806a-a3bcc70a6dfd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:50.406120 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525f9f4-821e-45a3-8c27-aef4720606ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:50.407476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f9f4-821e-45a3-8c27-aef4720606ff None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:50.408011 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f9f4-821e-45a3-8c27-aef4720606ff None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:50.408635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525f9f4-821e-45a3-8c27-aef4720606ff None admin] PATH_INFO: `/v3/projects/260dfcd27fc6476da82f72872627a534` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:50.410240 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525f9f4-821e-45a3-8c27-aef4720606ff None admin] RBAC: Authorizing `identity:get_project(project_id=260dfcd27fc6476da82f72872627a534)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:50.412210 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525f9f4-821e-45a3-8c27-aef4720606ff None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:50.414188 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 67/127] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:50 2022] GET /identity/v3/projects/260dfcd27fc6476da82f72872627a534 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:50.421077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525f9f4-821e-45a3-8c27-aef4720606ff None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:50.462467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:50.472229 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:50.516772 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8766f-d8ce-4598-806a-a3bcc70a6dfd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:50.545504 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525f9f4-821e-45a3-8c27-aef4720606ff None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:50.555399 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f5060-abfb-47a0-aca9-4dbf04328a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:50.556162 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f5060-abfb-47a0-aca9-4dbf04328a7d None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:50.556510 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f5060-abfb-47a0-aca9-4dbf04328a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:50.556882 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f5060-abfb-47a0-aca9-4dbf04328a7d None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47/users/40ad1f973cfa4a84aa61dd53e8afe1f6/roles/b589362f55584575bde80e8f639f66bb` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:50.558868 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f5060-abfb-47a0-aca9-4dbf04328a7d None admin] RBAC: Authorizing `identity:create_grant(project_id=77d8e654d0884c519a11a6471e3aee47, user_id=40ad1f973cfa4a84aa61dd53e8afe1f6, role_id=b589362f55584575bde80e8f639f66bb)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:50.560380 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f5060-abfb-47a0-aca9-4dbf04328a7d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:50.574639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 61/128] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:13:50 2022] PUT /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47/users/40ad1f973cfa4a84aa61dd53e8afe1f6/roles/b589362f55584575bde80e8f639f66bb => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:13:50.586813 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052b4ff-c0f1-4456-ade0-da67b6c51025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:50.587540 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b4ff-c0f1-4456-ade0-da67b6c51025 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:50.587830 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b4ff-c0f1-4456-ade0-da67b6c51025 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:50.588114 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b4ff-c0f1-4456-ade0-da67b6c51025 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:50.588943 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052b4ff-c0f1-4456-ade0-da67b6c51025 None admin] RBAC: Adding query filter params (role.id=6914eb62761341d7b8640a878225f115, scope.project.id=260dfcd27fc6476da82f72872627a534, user.id=a0147318766a4812b77983481386e463) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:50.589264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052b4ff-c0f1-4456-ade0-da67b6c51025 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:50.590930 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052b4ff-c0f1-4456-ade0-da67b6c51025 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:50.600166 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 68/129] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:13:50 2022] GET /identity/v3/role_assignments?user.id=a0147318766a4812b77983481386e463&scope.project.id=260dfcd27fc6476da82f72872627a534&role.id=6914eb62761341d7b8640a878225f115 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:51.858984 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6d38e-c68a-4391-8e36-284a027b01f7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:51.859491 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6d38e-c68a-4391-8e36-284a027b01f7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:51.859570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6d38e-c68a-4391-8e36-284a027b01f7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:51.860981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 62/130] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:51.866352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7b0cd-4fd2-4871-a9f8-def84bb186f3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:51.866770 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7b0cd-4fd2-4871-a9f8-def84bb186f3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:51.866915 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7b0cd-4fd2-4871-a9f8-def84bb186f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:51.885315 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-52b7b0cd-4fd2-4871-a9f8-def84bb186f3 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:51.886533 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-52b7b0cd-4fd2-4871-a9f8-def84bb186f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:51.953700 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 69/131] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:52.222541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6d38e-c68a-4391-8e36-284a027b01f7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:52.324718 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6d38e-c68a-4391-8e36-284a027b01f7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:52.350919 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:52.360021 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f423bd-3bd8-4e73-b3bb-1c45b7fef53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:52.360685 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f423bd-3bd8-4e73-b3bb-1c45b7fef53e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:52.360968 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f423bd-3bd8-4e73-b3bb-1c45b7fef53e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:52.361240 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f423bd-3bd8-4e73-b3bb-1c45b7fef53e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:52.362888 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f423bd-3bd8-4e73-b3bb-1c45b7fef53e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:52.364337 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f423bd-3bd8-4e73-b3bb-1c45b7fef53e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:52.365949 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 63/132] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:52.371423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7b0cd-4fd2-4871-a9f8-def84bb186f3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:52.396077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:52.469868 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7b0cd-4fd2-4871-a9f8-def84bb186f3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:52.503725 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a42f07-f011-411c-90f2-6f42eefcd5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:52.504276 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a42f07-f011-411c-90f2-6f42eefcd5d7 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:52.504584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a42f07-f011-411c-90f2-6f42eefcd5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:52.504864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a42f07-f011-411c-90f2-6f42eefcd5d7 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:52.505530 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a42f07-f011-411c-90f2-6f42eefcd5d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:52.506964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a42f07-f011-411c-90f2-6f42eefcd5d7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:52.528649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 70/133] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:13:52 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:13:52.834641 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a1093-5d28-4a4c-8499-767c96863d9c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:52.834859 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a1093-5d28-4a4c-8499-767c96863d9c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:52.835144 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a1093-5d28-4a4c-8499-767c96863d9c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:52.836894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 64/134] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:52.842194 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a72e4-7731-400d-a01b-55042e5673a2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:52.842482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a72e4-7731-400d-a01b-55042e5673a2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:52.842758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a72e4-7731-400d-a01b-55042e5673a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:52.861545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-aa0a72e4-7731-400d-a01b-55042e5673a2 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:52.863319 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-aa0a72e4-7731-400d-a01b-55042e5673a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:52.933957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 71/135] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:53.201578 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a1093-5d28-4a4c-8499-767c96863d9c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:53.308812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a1093-5d28-4a4c-8499-767c96863d9c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:53.345639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2a76d-95be-45c4-967b-ad69588c5557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:53.346281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2a76d-95be-45c4-967b-ad69588c5557 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:53.346617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2a76d-95be-45c4-967b-ad69588c5557 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:53.346901 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2a76d-95be-45c4-967b-ad69588c5557 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:53.357601 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2a76d-95be-45c4-967b-ad69588c5557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6dd2a76d-95be-45c4-967b-ad69588c5557 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:53.362999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:53.363862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:53.364679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 65/136] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:53.371543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a72e4-7731-400d-a01b-55042e5673a2 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:53.385144 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:53.476233 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a72e4-7731-400d-a01b-55042e5673a2 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:53.511814 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda241ca-3dc1-4489-a615-8764d80c0a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:53.512443 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda241ca-3dc1-4489-a615-8764d80c0a97 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:53.512758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda241ca-3dc1-4489-a615-8764d80c0a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:53.513040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda241ca-3dc1-4489-a615-8764d80c0a97 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:53.513716 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda241ca-3dc1-4489-a615-8764d80c0a97 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:53.514026 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda241ca-3dc1-4489-a615-8764d80c0a97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:53.515464 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda241ca-3dc1-4489-a615-8764d80c0a97 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:53.525051 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 72/137] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:13:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:53.531064 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2a76d-95be-45c4-967b-ad69588c5557 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:53.592324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:53.643961 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2a76d-95be-45c4-967b-ad69588c5557 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:53.681380 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c111ac-81a9-4801-b250-48f20dc6a83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:53.682036 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c111ac-81a9-4801-b250-48f20dc6a83f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:53.682318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c111ac-81a9-4801-b250-48f20dc6a83f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:53.682591 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c111ac-81a9-4801-b250-48f20dc6a83f None admin] PATH_INFO: `/v3/users/40ad1f973cfa4a84aa61dd53e8afe1f6` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:53.683757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c111ac-81a9-4801-b250-48f20dc6a83f None admin] RBAC: Authorizing `identity:get_user(user_id=40ad1f973cfa4a84aa61dd53e8afe1f6)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:53.685242 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c111ac-81a9-4801-b250-48f20dc6a83f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:53.686849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 66/138] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:53 2022] GET /identity/v3/users/40ad1f973cfa4a84aa61dd53e8afe1f6 => generated 289 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:53.692946 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda241ca-3dc1-4489-a615-8764d80c0a97 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:53.770946 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:53.803156 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda241ca-3dc1-4489-a615-8764d80c0a97 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:53.839840 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3b3f0-1014-4bd8-9eaa-a4295b549a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:53.840811 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3b3f0-1014-4bd8-9eaa-a4295b549a43 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:53.841333 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3b3f0-1014-4bd8-9eaa-a4295b549a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:53.841829 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3b3f0-1014-4bd8-9eaa-a4295b549a43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:53.910590 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3b3f0-1014-4bd8-9eaa-a4295b549a43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:53.912068 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3b3f0-1014-4bd8-9eaa-a4295b549a43 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:53.981205 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:54.011868 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 73/139] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:13:54.017411 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c111ac-81a9-4801-b250-48f20dc6a83f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:54.123876 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c111ac-81a9-4801-b250-48f20dc6a83f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:54.148397 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:54.159945 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb1526-c16a-4fb3-8cde-0651977b531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:54.160621 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb1526-c16a-4fb3-8cde-0651977b531b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:54.160905 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb1526-c16a-4fb3-8cde-0651977b531b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:54.161179 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb1526-c16a-4fb3-8cde-0651977b531b None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:54.162366 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb1526-c16a-4fb3-8cde-0651977b531b None admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:54.163802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb1526-c16a-4fb3-8cde-0651977b531b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:54.165679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 67/140] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:54 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:54.171368 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3b3f0-1014-4bd8-9eaa-a4295b549a43 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:54.273341 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3b3f0-1014-4bd8-9eaa-a4295b549a43 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:54.307893 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7bb5e-4b2d-42fd-b0f7-2b9c2f7a52a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:54.308512 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7bb5e-4b2d-42fd-b0f7-2b9c2f7a52a1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:54.308820 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7bb5e-4b2d-42fd-b0f7-2b9c2f7a52a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:54.309098 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7bb5e-4b2d-42fd-b0f7-2b9c2f7a52a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:54.309876 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7bb5e-4b2d-42fd-b0f7-2b9c2f7a52a1 None admin] RBAC: Adding query filter params (role.id=b589362f55584575bde80e8f639f66bb, scope.project.id=77d8e654d0884c519a11a6471e3aee47, user.id=40ad1f973cfa4a84aa61dd53e8afe1f6) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:54.310200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7bb5e-4b2d-42fd-b0f7-2b9c2f7a52a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:54.311655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7bb5e-4b2d-42fd-b0f7-2b9c2f7a52a1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:54.317871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 74/141] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:13:54 2022] GET /identity/v3/role_assignments?user.id=40ad1f973cfa4a84aa61dd53e8afe1f6&scope.project.id=77d8e654d0884c519a11a6471e3aee47&role.id=b589362f55584575bde80e8f639f66bb => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:54.847703 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6d50-557e-49f4-b7c9-13fcc79acd66 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:54.848221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6d50-557e-49f4-b7c9-13fcc79acd66 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:54.848301 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6d50-557e-49f4-b7c9-13fcc79acd66 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:54.849817 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 68/142] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:54.855011 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd5077-1113-4d82-8e50-c10ea940e856 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:54.855321 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd5077-1113-4d82-8e50-c10ea940e856 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:54.855567 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd5077-1113-4d82-8e50-c10ea940e856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:54.875675 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3ffd5077-1113-4d82-8e50-c10ea940e856 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:54.876927 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3ffd5077-1113-4d82-8e50-c10ea940e856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:54.910407 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:54.954795 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 75/143] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:54.983458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa7274-6571-43ec-a6f3-f216c001e84e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:54.983736 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa7274-6571-43ec-a6f3-f216c001e84e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:54.984007 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa7274-6571-43ec-a6f3-f216c001e84e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:54.985421 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 69/144] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:54.990585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20efa-a6cc-4ef6-9b83-a419901382b3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:54.991128 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20efa-a6cc-4ef6-9b83-a419901382b3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:54.991128 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20efa-a6cc-4ef6-9b83-a419901382b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:55.010178 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7ab20efa-a6cc-4ef6-9b83-a419901382b3 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:55.011358 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7ab20efa-a6cc-4ef6-9b83-a419901382b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:55.067524 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded018b-322c-4c04-8572-711fe0c1a1cb None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:55.068028 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded018b-322c-4c04-8572-711fe0c1a1cb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:55.068395 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded018b-322c-4c04-8572-711fe0c1a1cb None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:55.069844 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 70/145] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:13:55.074886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c999b-e68c-4142-b0a1-1e462a28f715 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:55.075170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c999b-e68c-4142-b0a1-1e462a28f715 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:55.075438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c999b-e68c-4142-b0a1-1e462a28f715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:55.080570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 76/146] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:55.097588 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b57c999b-e68c-4142-b0a1-1e462a28f715 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:55.098847 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b57c999b-e68c-4142-b0a1-1e462a28f715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:55.165691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 71/147] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:13:55.228275 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab20efa-a6cc-4ef6-9b83-a419901382b3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:55.321869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:55.328683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab20efa-a6cc-4ef6-9b83-a419901382b3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:55.347913 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c999b-e68c-4142-b0a1-1e462a28f715 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:55.365038 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0d77a-af73-4082-9553-fca7449a0876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:55.365603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0d77a-af73-4082-9553-fca7449a0876 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:55.365885 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0d77a-af73-4082-9553-fca7449a0876 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:55.366160 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0d77a-af73-4082-9553-fca7449a0876 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:55.376823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0d77a-af73-4082-9553-fca7449a0876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e6a0d77a-af73-4082-9553-fca7449a0876 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:55.382104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:55.382955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:13:55.383723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 77/148] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:13:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:55.388790 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0d77a-af73-4082-9553-fca7449a0876 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:55.463447 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c999b-e68c-4142-b0a1-1e462a28f715 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:55.485840 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:55.496322 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0d77a-af73-4082-9553-fca7449a0876 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:55.497752 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad101b-c4ba-4453-94c4-b79432927241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:55.498407 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad101b-c4ba-4453-94c4-b79432927241 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:55.498677 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad101b-c4ba-4453-94c4-b79432927241 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:55.498943 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad101b-c4ba-4453-94c4-b79432927241 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:55.509653 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad101b-c4ba-4453-94c4-b79432927241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-86ad101b-c4ba-4453-94c4-b79432927241 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:55.514430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:55.515483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 72/149] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:13:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:55.521151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad101b-c4ba-4453-94c4-b79432927241 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:55.530281 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1f0be-b8c5-4f02-b3f1-2cca4c9631d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:55.530887 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1f0be-b8c5-4f02-b3f1-2cca4c9631d9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:55.531164 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1f0be-b8c5-4f02-b3f1-2cca4c9631d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:55.531432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1f0be-b8c5-4f02-b3f1-2cca4c9631d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:55.532085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1f0be-b8c5-4f02-b3f1-2cca4c9631d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:55.532382 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1f0be-b8c5-4f02-b3f1-2cca4c9631d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:55.533787 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1f0be-b8c5-4f02-b3f1-2cca4c9631d9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:55.541630 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 78/150] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:55.542469 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1f0be-b8c5-4f02-b3f1-2cca4c9631d9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:55.562491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:55.621280 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad101b-c4ba-4453-94c4-b79432927241 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:55.644586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1f0be-b8c5-4f02-b3f1-2cca4c9631d9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:55.656735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bfb63-97ea-4c47-a5cf-8a5d8a9915e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:55.657370 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bfb63-97ea-4c47-a5cf-8a5d8a9915e2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:55.657656 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bfb63-97ea-4c47-a5cf-8a5d8a9915e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:55.657928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bfb63-97ea-4c47-a5cf-8a5d8a9915e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:55.659049 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bfb63-97ea-4c47-a5cf-8a5d8a9915e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:55.660527 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bfb63-97ea-4c47-a5cf-8a5d8a9915e2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:55.662141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 73/151] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:55.662141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bfb63-97ea-4c47-a5cf-8a5d8a9915e2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:55.678952 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a41c93-4bd8-49bc-8c32-66e92006c5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:55.679566 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a41c93-4bd8-49bc-8c32-66e92006c5ac None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:55.679848 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a41c93-4bd8-49bc-8c32-66e92006c5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:55.680123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a41c93-4bd8-49bc-8c32-66e92006c5ac None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:55.680820 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a41c93-4bd8-49bc-8c32-66e92006c5ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:55.681124 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a41c93-4bd8-49bc-8c32-66e92006c5ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:55.682510 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a41c93-4bd8-49bc-8c32-66e92006c5ac None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:55.690192 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 79/152] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:13:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:55.690999 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a41c93-4bd8-49bc-8c32-66e92006c5ac None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:55.714091 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:55.765601 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bfb63-97ea-4c47-a5cf-8a5d8a9915e2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:55.771104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:55.791114 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a41c93-4bd8-49bc-8c32-66e92006c5ac None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:55.798690 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671b08b-7ef2-42d0-a147-d058b43b82af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:55.799319 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671b08b-7ef2-42d0-a147-d058b43b82af None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:55.799610 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671b08b-7ef2-42d0-a147-d058b43b82af None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:55.800383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671b08b-7ef2-42d0-a147-d058b43b82af None admin] PATH_INFO: `/v3/users/c90b6b2d2929421d95063741e590bc77` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:55.812989 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671b08b-7ef2-42d0-a147-d058b43b82af None admin] RBAC: Authorizing `identity:get_user(user_id=c90b6b2d2929421d95063741e590bc77)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:55.814434 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671b08b-7ef2-42d0-a147-d058b43b82af None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:55.816889 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 74/153] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:55 2022] GET /identity/v3/users/c90b6b2d2929421d95063741e590bc77 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:55.816889 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671b08b-7ef2-42d0-a147-d058b43b82af None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:55.829476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956c6ce-d6a6-497a-ba33-a9c892915354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:55.830099 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c6ce-d6a6-497a-ba33-a9c892915354 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:55.830371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c6ce-d6a6-497a-ba33-a9c892915354 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:55.830641 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c6ce-d6a6-497a-ba33-a9c892915354 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:55.831347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956c6ce-d6a6-497a-ba33-a9c892915354 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:55.832804 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956c6ce-d6a6-497a-ba33-a9c892915354 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:55.854100 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 80/154] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:13:55 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:13:55.854914 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956c6ce-d6a6-497a-ba33-a9c892915354 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:55.941679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671b08b-7ef2-42d0-a147-d058b43b82af None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:55.952251 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:55.967509 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956c6ce-d6a6-497a-ba33-a9c892915354 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:55.982074 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cb33d-057b-43c9-ac8d-da6258ca743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:55.982738 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cb33d-057b-43c9-ac8d-da6258ca743c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:55.983010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cb33d-057b-43c9-ac8d-da6258ca743c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:55.983288 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cb33d-057b-43c9-ac8d-da6258ca743c None admin] PATH_INFO: `/v3/users/c90b6b2d2929421d95063741e590bc77` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:55.984498 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cb33d-057b-43c9-ac8d-da6258ca743c None admin] RBAC: Authorizing `identity:get_user(user_id=c90b6b2d2929421d95063741e590bc77)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:55.985998 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cb33d-057b-43c9-ac8d-da6258ca743c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:55.988215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 75/155] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:55 2022] GET /identity/v3/users/c90b6b2d2929421d95063741e590bc77 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:55.993608 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cb33d-057b-43c9-ac8d-da6258ca743c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:56.010856 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa714c-6583-4cba-bc36-7ba2dc2e2ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:56.011512 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa714c-6583-4cba-bc36-7ba2dc2e2ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:56.011788 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa714c-6583-4cba-bc36-7ba2dc2e2ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:56.012063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa714c-6583-4cba-bc36-7ba2dc2e2ca4 None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:56.013225 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa714c-6583-4cba-bc36-7ba2dc2e2ca4 None admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:56.015896 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa714c-6583-4cba-bc36-7ba2dc2e2ca4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:56.023061 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 81/156] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:55 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:56.024342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa714c-6583-4cba-bc36-7ba2dc2e2ca4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:56.028828 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:56.124995 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cb33d-057b-43c9-ac8d-da6258ca743c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:56.149269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa714c-6583-4cba-bc36-7ba2dc2e2ca4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:56.170433 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af123c-4555-4314-8edb-b102ad5bb1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:56.171080 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af123c-4555-4314-8edb-b102ad5bb1aa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:56.171359 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af123c-4555-4314-8edb-b102ad5bb1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:56.171640 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af123c-4555-4314-8edb-b102ad5bb1aa None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:56.173621 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af123c-4555-4314-8edb-b102ad5bb1aa None admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:56.174299 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af123c-4555-4314-8edb-b102ad5bb1aa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:56.176716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 76/157] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:55 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:56.183228 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af123c-4555-4314-8edb-b102ad5bb1aa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:56.197845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dc789-e896-4177-b343-3872b0b8df36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:56.198524 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc789-e896-4177-b343-3872b0b8df36 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:56.198807 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc789-e896-4177-b343-3872b0b8df36 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:56.199136 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc789-e896-4177-b343-3872b0b8df36 None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f/users/c90b6b2d2929421d95063741e590bc77/roles/81d287a8b606458d87c9607e8b48ac58` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:56.201157 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dc789-e896-4177-b343-3872b0b8df36 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7512633e55e45dcb163a688cd54156f, user_id=c90b6b2d2929421d95063741e590bc77, role_id=81d287a8b606458d87c9607e8b48ac58)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:56.202731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dc789-e896-4177-b343-3872b0b8df36 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:56.218080 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 82/158] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:13:56 2022] PUT /identity/v3/projects/c7512633e55e45dcb163a688cd54156f/users/c90b6b2d2929421d95063741e590bc77/roles/81d287a8b606458d87c9607e8b48ac58 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:13:56.238433 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:56.307690 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af123c-4555-4314-8edb-b102ad5bb1aa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:56.350257 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ca164-5d66-471c-821d-3170787cc695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:56.351325 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ca164-5d66-471c-821d-3170787cc695 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:56.351925 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ca164-5d66-471c-821d-3170787cc695 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:56.352405 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ca164-5d66-471c-821d-3170787cc695 None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f/users/c90b6b2d2929421d95063741e590bc77/roles/1255151437d94364acba841f4215321c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:56.355009 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ca164-5d66-471c-821d-3170787cc695 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7512633e55e45dcb163a688cd54156f, user_id=c90b6b2d2929421d95063741e590bc77, role_id=1255151437d94364acba841f4215321c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:56.356908 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ca164-5d66-471c-821d-3170787cc695 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:56.373169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 77/159] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:13:56 2022] PUT /identity/v3/projects/c7512633e55e45dcb163a688cd54156f/users/c90b6b2d2929421d95063741e590bc77/roles/1255151437d94364acba841f4215321c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:13:58.285721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d25e-7786-497b-8dc4-3569e6a09f2a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:58.286249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d25e-7786-497b-8dc4-3569e6a09f2a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:58.286249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d25e-7786-497b-8dc4-3569e6a09f2a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:58.287745 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 83/160] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:58.293237 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45ddad-0dea-462c-af03-69dc96c84234 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:58.293976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45ddad-0dea-462c-af03-69dc96c84234 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:58.294385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45ddad-0dea-462c-af03-69dc96c84234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:58.313866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4c45ddad-0dea-462c-af03-69dc96c84234 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:58.315838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4c45ddad-0dea-462c-af03-69dc96c84234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:58.356890 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331386-809d-4a90-85b1-a2eae7254a13 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:58.357505 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331386-809d-4a90-85b1-a2eae7254a13 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:58.357973 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f331386-809d-4a90-85b1-a2eae7254a13 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:58.360092 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 84/161] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:13:58.365811 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3367836-22c4-4f6f-9f89-b70858c1f320 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:58.366335 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3367836-22c4-4f6f-9f89-b70858c1f320 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:58.366800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3367836-22c4-4f6f-9f89-b70858c1f320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:58.386303 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b3367836-22c4-4f6f-9f89-b70858c1f320 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:58.387449 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b3367836-22c4-4f6f-9f89-b70858c1f320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:58.390429 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 78/162] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:58.450536 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:58.462143 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 85/163] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:58.552635 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c870-69f7-41cc-9d08-4769b22b07f8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:58.553238 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c870-69f7-41cc-9d08-4769b22b07f8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:58.553723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c870-69f7-41cc-9d08-4769b22b07f8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:58.555288 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 79/164] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:13:58.561927 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6241cb6-89fc-4966-b767-bf458dcc3cdf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:58.562216 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6241cb6-89fc-4966-b767-bf458dcc3cdf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:58.562495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6241cb6-89fc-4966-b767-bf458dcc3cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:58.581501 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b6241cb6-89fc-4966-b767-bf458dcc3cdf None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:58.582697 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b6241cb6-89fc-4966-b767-bf458dcc3cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:58.605634 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465ff96-ac8c-4abd-9014-d26052dae68c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:58.605914 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465ff96-ac8c-4abd-9014-d26052dae68c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:58.606357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465ff96-ac8c-4abd-9014-d26052dae68c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:58.607868 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 80/165] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:13:58.613157 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3dbe-8256-409f-a287-4ebde48f23f5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:58.613438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3dbe-8256-409f-a287-4ebde48f23f5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:58.613714 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3dbe-8256-409f-a287-4ebde48f23f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:58.631585 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-fdce3dbe-8256-409f-a287-4ebde48f23f5 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:13:58.632809 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fdce3dbe-8256-409f-a287-4ebde48f23f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:13:58.651931 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 86/166] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:58.680989 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:58.686381 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6241cb6-89fc-4966-b767-bf458dcc3cdf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:58.719365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 81/167] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:13:58.735862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce3dbe-8256-409f-a287-4ebde48f23f5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:58.832355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6241cb6-89fc-4966-b767-bf458dcc3cdf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:58.844954 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce3dbe-8256-409f-a287-4ebde48f23f5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:58.857893 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:58.870026 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5b3d2-b5d3-4a45-9403-b7494b063997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:58.870656 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5b3d2-b5d3-4a45-9403-b7494b063997 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:58.870932 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5b3d2-b5d3-4a45-9403-b7494b063997 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:58.871202 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5b3d2-b5d3-4a45-9403-b7494b063997 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:58.882496 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5b3d2-b5d3-4a45-9403-b7494b063997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-47b5b3d2-b5d3-4a45-9403-b7494b063997 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:58.887687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:58.888671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:13:58.889438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 87/168] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:13:58.891993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e4cd0-7877-4913-9ef2-0f7a0bf24a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:58.892666 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e4cd0-7877-4913-9ef2-0f7a0bf24a43 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:58.892956 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e4cd0-7877-4913-9ef2-0f7a0bf24a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:58.893213 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e4cd0-7877-4913-9ef2-0f7a0bf24a43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:58.894395 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e4cd0-7877-4913-9ef2-0f7a0bf24a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:58.894775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5b3d2-b5d3-4a45-9403-b7494b063997 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:58.895835 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e4cd0-7877-4913-9ef2-0f7a0bf24a43 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:58.897452 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 82/169] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:13:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:58.903181 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e4cd0-7877-4913-9ef2-0f7a0bf24a43 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:58.936592 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:59.010001 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e4cd0-7877-4913-9ef2-0f7a0bf24a43 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:59.019013 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5b3d2-b5d3-4a45-9403-b7494b063997 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:59.042791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dc705-9c0a-4a5d-9ca9-ced7e072b4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:59.043391 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dc705-9c0a-4a5d-9ca9-ced7e072b4bc None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:59.043664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dc705-9c0a-4a5d-9ca9-ced7e072b4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:59.043936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dc705-9c0a-4a5d-9ca9-ced7e072b4bc None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:59.044600 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dc705-9c0a-4a5d-9ca9-ced7e072b4bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:59.046020 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dc705-9c0a-4a5d-9ca9-ced7e072b4bc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:59.051192 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0f880-06c0-4691-a3c7-f8fb9fa3389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:59.051793 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0f880-06c0-4691-a3c7-f8fb9fa3389a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:59.052069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0f880-06c0-4691-a3c7-f8fb9fa3389a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:59.052336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0f880-06c0-4691-a3c7-f8fb9fa3389a None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:59.053022 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0f880-06c0-4691-a3c7-f8fb9fa3389a None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:59.053321 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0f880-06c0-4691-a3c7-f8fb9fa3389a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:59.054706 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0f880-06c0-4691-a3c7-f8fb9fa3389a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:59.062694 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 88/170] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:13:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:13:59.063476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0f880-06c0-4691-a3c7-f8fb9fa3389a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:59.067234 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 83/171] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:13:58 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:13:59.068192 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dc705-9c0a-4a5d-9ca9-ced7e072b4bc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:59.118409 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:59.170912 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:59.181629 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0f880-06c0-4691-a3c7-f8fb9fa3389a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:59.186316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dc705-9c0a-4a5d-9ca9-ced7e072b4bc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:59.218676 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3c855-413a-4346-bf0f-65c99490fe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:59.219327 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3c855-413a-4346-bf0f-65c99490fe02 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:59.219614 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3c855-413a-4346-bf0f-65c99490fe02 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:59.219890 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3c855-413a-4346-bf0f-65c99490fe02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:59.232200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3c855-413a-4346-bf0f-65c99490fe02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:59.232791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261346-dcd8-46b1-a5cf-923581b8bec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:59.233653 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261346-dcd8-46b1-a5cf-923581b8bec4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:59.234154 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261346-dcd8-46b1-a5cf-923581b8bec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:59.234614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261346-dcd8-46b1-a5cf-923581b8bec4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4de3c855-413a-4346-bf0f-65c99490fe02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:59.238133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:59.239123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 89/172] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:13:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:59.240086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3c855-413a-4346-bf0f-65c99490fe02 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:59.254469 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff261346-dcd8-46b1-a5cf-923581b8bec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ff261346-dcd8-46b1-a5cf-923581b8bec4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:13:59.260927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:13:59.261786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:13:59.262539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:13:59.264867 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 84/173] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:13:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:13:59.265903 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261346-dcd8-46b1-a5cf-923581b8bec4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:59.361032 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3c855-413a-4346-bf0f-65c99490fe02 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:59.369096 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:59.399131 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261346-dcd8-46b1-a5cf-923581b8bec4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:59.407225 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3472489-d27c-42a7-aa84-c15225c51f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:59.407869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3472489-d27c-42a7-aa84-c15225c51f26 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:59.408147 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3472489-d27c-42a7-aa84-c15225c51f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:59.408414 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3472489-d27c-42a7-aa84-c15225c51f26 None admin] PATH_INFO: `/v3/users/6cdd0bbd1eab42f89b3e480afb71beff` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:59.430046 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3472489-d27c-42a7-aa84-c15225c51f26 None admin] RBAC: Authorizing `identity:get_user(user_id=6cdd0bbd1eab42f89b3e480afb71beff)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:59.431517 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3472489-d27c-42a7-aa84-c15225c51f26 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:59.433223 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 90/174] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:59 2022] GET /identity/v3/users/6cdd0bbd1eab42f89b3e480afb71beff => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:59.438451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3472489-d27c-42a7-aa84-c15225c51f26 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:59.439782 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d2eb2-dac2-42d4-9f1c-7a03241611d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:59.440555 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d2eb2-dac2-42d4-9f1c-7a03241611d0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:59.440846 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d2eb2-dac2-42d4-9f1c-7a03241611d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:59.441122 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d2eb2-dac2-42d4-9f1c-7a03241611d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:59.441799 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d2eb2-dac2-42d4-9f1c-7a03241611d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:59.442104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d2eb2-dac2-42d4-9f1c-7a03241611d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:59.443501 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d2eb2-dac2-42d4-9f1c-7a03241611d0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:59.452319 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 85/175] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:59.453191 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d2eb2-dac2-42d4-9f1c-7a03241611d0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:59.472988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:59.552977 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3472489-d27c-42a7-aa84-c15225c51f26 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:59.579786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d2eb2-dac2-42d4-9f1c-7a03241611d0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:59.590419 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cad18-13b8-46e5-a0c1-e42282a42329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:59.591045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cad18-13b8-46e5-a0c1-e42282a42329 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:59.591328 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cad18-13b8-46e5-a0c1-e42282a42329 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:59.591601 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cad18-13b8-46e5-a0c1-e42282a42329 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:59.592272 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cad18-13b8-46e5-a0c1-e42282a42329 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:13:59.592599 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cad18-13b8-46e5-a0c1-e42282a42329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:59.593994 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cad18-13b8-46e5-a0c1-e42282a42329 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:59.602810 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 91/176] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:13:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:59.603460 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cad18-13b8-46e5-a0c1-e42282a42329 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:59.622969 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c6b7a-b4ce-4e25-a592-3ef7e364a323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:59.623966 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c6b7a-b4ce-4e25-a592-3ef7e364a323 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:59.624458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c6b7a-b4ce-4e25-a592-3ef7e364a323 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:59.625022 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c6b7a-b4ce-4e25-a592-3ef7e364a323 None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:59.626523 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c6b7a-b4ce-4e25-a592-3ef7e364a323 None admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:59.628747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c6b7a-b4ce-4e25-a592-3ef7e364a323 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:59.631794 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 86/177] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:13:59 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:59.637006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:59.637830 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c6b7a-b4ce-4e25-a592-3ef7e364a323 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:59.720902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cad18-13b8-46e5-a0c1-e42282a42329 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:59.757588 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20117a53-8e6d-47ae-a650-6ba6449f6328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:59.758583 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20117a53-8e6d-47ae-a650-6ba6449f6328 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:59.759065 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20117a53-8e6d-47ae-a650-6ba6449f6328 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:59.759561 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20117a53-8e6d-47ae-a650-6ba6449f6328 None admin] PATH_INFO: `/v3/users/c90b6b2d2929421d95063741e590bc77` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:59.760989 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20117a53-8e6d-47ae-a650-6ba6449f6328 None admin] RBAC: Authorizing `identity:get_user(user_id=c90b6b2d2929421d95063741e590bc77)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:59.761118 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:59.763043 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20117a53-8e6d-47ae-a650-6ba6449f6328 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:59.765667 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 92/178] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:59 2022] GET /identity/v3/users/c90b6b2d2929421d95063741e590bc77 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:59.766857 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20117a53-8e6d-47ae-a650-6ba6449f6328 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:59.790317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c6b7a-b4ce-4e25-a592-3ef7e364a323 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:59.838082 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f9997-361c-4aff-9dcd-6271caac99cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:59.838961 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f9997-361c-4aff-9dcd-6271caac99cb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:59.839258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f9997-361c-4aff-9dcd-6271caac99cb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:59.839532 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f9997-361c-4aff-9dcd-6271caac99cb None admin] PATH_INFO: `/v3/users/c90b6b2d2929421d95063741e590bc77` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:59.841304 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f9997-361c-4aff-9dcd-6271caac99cb None admin] RBAC: Authorizing `identity:get_user(user_id=c90b6b2d2929421d95063741e590bc77)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:59.843060 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f9997-361c-4aff-9dcd-6271caac99cb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:59.845871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 87/179] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:13:59 2022] GET /identity/v3/users/c90b6b2d2929421d95063741e590bc77 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:13:59.846689 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f9997-361c-4aff-9dcd-6271caac99cb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:13:59.872032 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:13:59.881271 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20117a53-8e6d-47ae-a650-6ba6449f6328 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:13:59.917578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162ab3-4407-464e-95df-b392600f40e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:13:59.918655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162ab3-4407-464e-95df-b392600f40e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:13:59.919130 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162ab3-4407-464e-95df-b392600f40e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:13:59.919662 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162ab3-4407-464e-95df-b392600f40e5 None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f/users/6cdd0bbd1eab42f89b3e480afb71beff/roles/6914eb62761341d7b8640a878225f115` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:13:59.921769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e162ab3-4407-464e-95df-b392600f40e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7512633e55e45dcb163a688cd54156f, user_id=6cdd0bbd1eab42f89b3e480afb71beff, role_id=6914eb62761341d7b8640a878225f115)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:13:59.923488 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e162ab3-4407-464e-95df-b392600f40e5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:13:59.944798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 93/180] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:13:59 2022] PUT /identity/v3/projects/c7512633e55e45dcb163a688cd54156f/users/6cdd0bbd1eab42f89b3e480afb71beff/roles/6914eb62761341d7b8640a878225f115 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:13:59.947089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162ab3-4407-464e-95df-b392600f40e5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:00.016916 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f9997-361c-4aff-9dcd-6271caac99cb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:00.090836 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436631d-1d87-46d9-a04d-49555a116c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:00.093099 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436631d-1d87-46d9-a04d-49555a116c6e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:00.093385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436631d-1d87-46d9-a04d-49555a116c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:00.093656 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436631d-1d87-46d9-a04d-49555a116c6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:00.129189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:00.163298 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162ab3-4407-464e-95df-b392600f40e5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:00.208576 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436631d-1d87-46d9-a04d-49555a116c6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:00.210071 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c81e5-9ae4-4337-b7fb-b99468d7343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:00.210223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436631d-1d87-46d9-a04d-49555a116c6e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:00.211320 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c81e5-9ae4-4337-b7fb-b99468d7343a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:00.211806 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c81e5-9ae4-4337-b7fb-b99468d7343a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:00.212671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c81e5-9ae4-4337-b7fb-b99468d7343a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:00.247818 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:00.306552 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c81e5-9ae4-4337-b7fb-b99468d7343a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:00.308852 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c81e5-9ae4-4337-b7fb-b99468d7343a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:00.328522 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 88/181] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:13:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:00.334486 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436631d-1d87-46d9-a04d-49555a116c6e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:00.383566 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:00.438299 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436631d-1d87-46d9-a04d-49555a116c6e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:00.446622 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 94/182] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:13:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:00.454134 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c81e5-9ae4-4337-b7fb-b99468d7343a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:00.497543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16763c-082b-45a7-b464-51c227d22461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:00.498190 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16763c-082b-45a7-b464-51c227d22461 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:00.498748 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16763c-082b-45a7-b464-51c227d22461 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:00.498748 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16763c-082b-45a7-b464-51c227d22461 None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:00.499861 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16763c-082b-45a7-b464-51c227d22461 None admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:00.501448 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16763c-082b-45a7-b464-51c227d22461 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:00.503532 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 89/183] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:00 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:00.509319 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16763c-082b-45a7-b464-51c227d22461 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:00.519642 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:00.562230 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c81e5-9ae4-4337-b7fb-b99468d7343a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:00.599687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f942fb-a4d2-41e9-aa9c-6b345296491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:00.600691 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f942fb-a4d2-41e9-aa9c-6b345296491b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:00.601233 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f942fb-a4d2-41e9-aa9c-6b345296491b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:00.601789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f942fb-a4d2-41e9-aa9c-6b345296491b None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:00.603241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f942fb-a4d2-41e9-aa9c-6b345296491b None admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:00.604915 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f942fb-a4d2-41e9-aa9c-6b345296491b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:00.606765 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 95/184] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:00 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:00.612889 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f942fb-a4d2-41e9-aa9c-6b345296491b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:00.678482 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16763c-082b-45a7-b464-51c227d22461 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:00.704322 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:00.730508 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f942fb-a4d2-41e9-aa9c-6b345296491b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:00.755271 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:00.766467 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f4483-028b-451e-9ce4-929858fc52bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:00.767175 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f4483-028b-451e-9ce4-929858fc52bf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:00.767382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f4483-028b-451e-9ce4-929858fc52bf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:00.767615 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f4483-028b-451e-9ce4-929858fc52bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:00.768942 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f4483-028b-451e-9ce4-929858fc52bf None admin] RBAC: Adding query filter params (role.id=81d287a8b606458d87c9607e8b48ac58, scope.project.id=c7512633e55e45dcb163a688cd54156f, user.id=c90b6b2d2929421d95063741e590bc77) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:00.771107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f4483-028b-451e-9ce4-929858fc52bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:00.773936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f4483-028b-451e-9ce4-929858fc52bf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:00.780893 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50853c8-6135-4421-957b-5799ff2d975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:00.781507 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50853c8-6135-4421-957b-5799ff2d975c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:00.781783 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50853c8-6135-4421-957b-5799ff2d975c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:00.782050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50853c8-6135-4421-957b-5799ff2d975c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:00.782772 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50853c8-6135-4421-957b-5799ff2d975c None admin] RBAC: Adding query filter params (role.id=1255151437d94364acba841f4215321c, scope.project.id=c7512633e55e45dcb163a688cd54156f, user.id=c90b6b2d2929421d95063741e590bc77) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:00.783116 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50853c8-6135-4421-957b-5799ff2d975c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:00.784629 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50853c8-6135-4421-957b-5799ff2d975c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:00.788670 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 90/185] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:00 2022] GET /identity/v3/role_assignments?user.id=c90b6b2d2929421d95063741e590bc77&scope.project.id=c7512633e55e45dcb163a688cd54156f&role.id=81d287a8b606458d87c9607e8b48ac58 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:00.794107 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 96/186] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:00 2022] GET /identity/v3/role_assignments?user.id=c90b6b2d2929421d95063741e590bc77&scope.project.id=c7512633e55e45dcb163a688cd54156f&role.id=1255151437d94364acba841f4215321c => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:01.510628 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a24299-121d-4009-92ae-3c8fa80f895e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:01.511179 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a24299-121d-4009-92ae-3c8fa80f895e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:01.511179 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a24299-121d-4009-92ae-3c8fa80f895e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:01.512687 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 91/187] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:01.517641 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a809b-5fb4-4046-a236-ac1ff93e9817 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:01.518054 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a809b-5fb4-4046-a236-ac1ff93e9817 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:01.518231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a809b-5fb4-4046-a236-ac1ff93e9817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:01.537134 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a66a809b-5fb4-4046-a236-ac1ff93e9817 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:01.538325 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a66a809b-5fb4-4046-a236-ac1ff93e9817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:01.609197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 97/188] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:01.709910 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323748ce-c8b7-46fc-8ead-8cbaeec5ff27 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:01.710238 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323748ce-c8b7-46fc-8ead-8cbaeec5ff27 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:01.710459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323748ce-c8b7-46fc-8ead-8cbaeec5ff27 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:01.712191 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 92/189] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:01.717282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeb8e3-fdec-4e9f-90e0-c86377a8c94c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:01.717567 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeb8e3-fdec-4e9f-90e0-c86377a8c94c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:01.717834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeb8e3-fdec-4e9f-90e0-c86377a8c94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:01.735866 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-42aeb8e3-fdec-4e9f-90e0-c86377a8c94c None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:01.737052 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-42aeb8e3-fdec-4e9f-90e0-c86377a8c94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:01.804896 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 98/190] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:01.874089 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323748ce-c8b7-46fc-8ead-8cbaeec5ff27 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:01.976502 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323748ce-c8b7-46fc-8ead-8cbaeec5ff27 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:02.009601 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15116c-1a56-4a95-8346-a3033ab7464f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:02.010233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15116c-1a56-4a95-8346-a3033ab7464f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:02.010511 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15116c-1a56-4a95-8346-a3033ab7464f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:02.010779 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15116c-1a56-4a95-8346-a3033ab7464f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:02.021034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15116c-1a56-4a95-8346-a3033ab7464f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5d15116c-1a56-4a95-8346-a3033ab7464f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:02.026483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:02.027342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:02.028119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 93/191] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:02.035147 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeb8e3-fdec-4e9f-90e0-c86377a8c94c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:02.070496 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15116c-1a56-4a95-8346-a3033ab7464f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:02.084310 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:02.116213 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:02.138720 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeb8e3-fdec-4e9f-90e0-c86377a8c94c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:02.172763 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00304f5-6340-406c-bbba-b2ddec5c7e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:02.173378 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00304f5-6340-406c-bbba-b2ddec5c7e66 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:02.173653 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00304f5-6340-406c-bbba-b2ddec5c7e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:02.173917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00304f5-6340-406c-bbba-b2ddec5c7e66 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:02.174598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00304f5-6340-406c-bbba-b2ddec5c7e66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:02.174928 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00304f5-6340-406c-bbba-b2ddec5c7e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:02.176323 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00304f5-6340-406c-bbba-b2ddec5c7e66 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:02.183559 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15116c-1a56-4a95-8346-a3033ab7464f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:02.184161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 99/192] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:02.189669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00304f5-6340-406c-bbba-b2ddec5c7e66 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:02.220749 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16182bd4-05be-456e-bd88-c3928592c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:02.222134 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182bd4-05be-456e-bd88-c3928592c992 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:02.222556 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182bd4-05be-456e-bd88-c3928592c992 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:02.223097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182bd4-05be-456e-bd88-c3928592c992 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:02.224317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16182bd4-05be-456e-bd88-c3928592c992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:02.225855 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16182bd4-05be-456e-bd88-c3928592c992 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:02.228565 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 94/193] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:02.232765 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16182bd4-05be-456e-bd88-c3928592c992 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:02.296188 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00304f5-6340-406c-bbba-b2ddec5c7e66 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:02.306891 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:02.334090 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5c1c2-cbe0-46c5-9846-2527d6ac4334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:02.334659 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5c1c2-cbe0-46c5-9846-2527d6ac4334 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:02.334935 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5c1c2-cbe0-46c5-9846-2527d6ac4334 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:02.335212 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5c1c2-cbe0-46c5-9846-2527d6ac4334 None admin] PATH_INFO: `/v3/users/f760f433bfa14de49fe730653587a515` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:02.350418 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16182bd4-05be-456e-bd88-c3928592c992 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:02.351375 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5c1c2-cbe0-46c5-9846-2527d6ac4334 None admin] RBAC: Authorizing `identity:get_user(user_id=f760f433bfa14de49fe730653587a515)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:02.352865 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5c1c2-cbe0-46c5-9846-2527d6ac4334 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:02.354506 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 100/194] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:02 2022] GET /identity/v3/users/f760f433bfa14de49fe730653587a515 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:02.358718 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:02.359799 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5c1c2-cbe0-46c5-9846-2527d6ac4334 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:02.390671 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720c881-b58a-497a-8954-32829891ffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:02.391546 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720c881-b58a-497a-8954-32829891ffa9 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:02.391991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720c881-b58a-497a-8954-32829891ffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:02.392341 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720c881-b58a-497a-8954-32829891ffa9 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:02.392935 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720c881-b58a-497a-8954-32829891ffa9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:02.394894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720c881-b58a-497a-8954-32829891ffa9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:02.417139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 95/195] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:02 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:14:02.423861 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e4897-0e8c-43a0-9472-02b9369088e9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:02.424170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e4897-0e8c-43a0-9472-02b9369088e9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:02.424486 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e4897-0e8c-43a0-9472-02b9369088e9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:02.426115 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 96/196] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:02.431783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd55ecb-c9ed-4b69-bddd-7824d2dee3ee None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:02.432114 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd55ecb-c9ed-4b69-bddd-7824d2dee3ee None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:02.432354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd55ecb-c9ed-4b69-bddd-7824d2dee3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:02.452143 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9bd55ecb-c9ed-4b69-bddd-7824d2dee3ee None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:02.453551 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9bd55ecb-c9ed-4b69-bddd-7824d2dee3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:02.468713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5c1c2-cbe0-46c5-9846-2527d6ac4334 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:02.505631 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4bfa1-5f12-497f-994f-08b78748f900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:02.506285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4bfa1-5f12-497f-994f-08b78748f900 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:02.506565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4bfa1-5f12-497f-994f-08b78748f900 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:02.506834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4bfa1-5f12-497f-994f-08b78748f900 None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:02.508007 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4bfa1-5f12-497f-994f-08b78748f900 None admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:02.509518 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4bfa1-5f12-497f-994f-08b78748f900 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:02.511261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 101/197] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:02 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:02.517001 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4bfa1-5f12-497f-994f-08b78748f900 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:02.540146 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 97/198] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:02.552286 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:02.625898 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4bfa1-5f12-497f-994f-08b78748f900 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:02.665465 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c0937-9c5a-4ea0-9246-022952a4434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:02.666128 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c0937-9c5a-4ea0-9246-022952a4434d None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:02.666406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c0937-9c5a-4ea0-9246-022952a4434d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:02.666679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c0937-9c5a-4ea0-9246-022952a4434d None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f/users/f760f433bfa14de49fe730653587a515/roles/b589362f55584575bde80e8f639f66bb` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:02.668660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c0937-9c5a-4ea0-9246-022952a4434d None admin] RBAC: Authorizing `identity:create_grant(project_id=c7512633e55e45dcb163a688cd54156f, user_id=f760f433bfa14de49fe730653587a515, role_id=b589362f55584575bde80e8f639f66bb)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:02.670028 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c0937-9c5a-4ea0-9246-022952a4434d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:02.682558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 102/199] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:02 2022] PUT /identity/v3/projects/c7512633e55e45dcb163a688cd54156f/users/f760f433bfa14de49fe730653587a515/roles/b589362f55584575bde80e8f639f66bb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:14:02.818413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd55ecb-c9ed-4b69-bddd-7824d2dee3ee None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:02.923331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:02.932953 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd55ecb-c9ed-4b69-bddd-7824d2dee3ee None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:02.969181 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821a973-6843-4b26-9b0a-6047af8074a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:02.969825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821a973-6843-4b26-9b0a-6047af8074a0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:02.970105 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821a973-6843-4b26-9b0a-6047af8074a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:02.970380 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821a973-6843-4b26-9b0a-6047af8074a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:02.981454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821a973-6843-4b26-9b0a-6047af8074a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f821a973-6843-4b26-9b0a-6047af8074a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:02.986445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:02.987385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:02.988353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 98/200] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:02.995408 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c0937-9c5a-4ea0-9246-022952a4434d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:03.099435 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:03.107273 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c0937-9c5a-4ea0-9246-022952a4434d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:03.144302 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475cd2d-2209-458a-9ce8-808d4635d77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:03.144949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475cd2d-2209-458a-9ce8-808d4635d77d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:03.145230 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475cd2d-2209-458a-9ce8-808d4635d77d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:03.145501 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475cd2d-2209-458a-9ce8-808d4635d77d None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:03.146165 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475cd2d-2209-458a-9ce8-808d4635d77d None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:03.146466 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475cd2d-2209-458a-9ce8-808d4635d77d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:03.147910 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475cd2d-2209-458a-9ce8-808d4635d77d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:03.156785 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 103/201] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:03.163418 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821a973-6843-4b26-9b0a-6047af8074a0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:03.273156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821a973-6843-4b26-9b0a-6047af8074a0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:03.314703 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379718cf-9cd9-4775-9bc4-d53307d0c709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:03.315378 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379718cf-9cd9-4775-9bc4-d53307d0c709 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:03.315656 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379718cf-9cd9-4775-9bc4-d53307d0c709 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:03.315929 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379718cf-9cd9-4775-9bc4-d53307d0c709 None admin] PATH_INFO: `/v3/users/6cdd0bbd1eab42f89b3e480afb71beff` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:03.317175 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379718cf-9cd9-4775-9bc4-d53307d0c709 None admin] RBAC: Authorizing `identity:get_user(user_id=6cdd0bbd1eab42f89b3e480afb71beff)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:03.318687 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379718cf-9cd9-4775-9bc4-d53307d0c709 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:03.320591 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 99/202] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:03 2022] GET /identity/v3/users/6cdd0bbd1eab42f89b3e480afb71beff => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:03.325837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475cd2d-2209-458a-9ce8-808d4635d77d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:03.432213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475cd2d-2209-458a-9ce8-808d4635d77d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:03.468238 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb842e9-8d11-468d-8ad3-291b4bf05e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:03.468940 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb842e9-8d11-468d-8ad3-291b4bf05e05 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:03.469227 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb842e9-8d11-468d-8ad3-291b4bf05e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:03.469503 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb842e9-8d11-468d-8ad3-291b4bf05e05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:03.497830 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:03.548223 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb842e9-8d11-468d-8ad3-291b4bf05e05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:03.549743 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb842e9-8d11-468d-8ad3-291b4bf05e05 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:03.658038 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 104/203] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:03.664897 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379718cf-9cd9-4775-9bc4-d53307d0c709 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:03.690004 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:03.774382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379718cf-9cd9-4775-9bc4-d53307d0c709 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:03.810321 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbda85-131b-43bf-9c7a-5dc693c28f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:03.811227 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbda85-131b-43bf-9c7a-5dc693c28f5e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:03.811722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbda85-131b-43bf-9c7a-5dc693c28f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:03.812218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbda85-131b-43bf-9c7a-5dc693c28f5e None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:03.813570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbda85-131b-43bf-9c7a-5dc693c28f5e None admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:03.815255 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbda85-131b-43bf-9c7a-5dc693c28f5e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:03.817132 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 100/204] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:03 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:03.823165 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb842e9-8d11-468d-8ad3-291b4bf05e05 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:03.883227 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:03.931036 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb842e9-8d11-468d-8ad3-291b4bf05e05 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:03.967619 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351fef4-6153-4c55-983a-9719894bdb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:03.968586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351fef4-6153-4c55-983a-9719894bdb1e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:03.969109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351fef4-6153-4c55-983a-9719894bdb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:03.969589 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351fef4-6153-4c55-983a-9719894bdb1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:03.970502 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351fef4-6153-4c55-983a-9719894bdb1e None admin] RBAC: Adding query filter params (role.id=6914eb62761341d7b8640a878225f115, scope.project.id=c7512633e55e45dcb163a688cd54156f, user.id=6cdd0bbd1eab42f89b3e480afb71beff) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:03.971010 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351fef4-6153-4c55-983a-9719894bdb1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:03.972828 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351fef4-6153-4c55-983a-9719894bdb1e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:03.979096 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 105/205] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:03 2022] GET /identity/v3/role_assignments?user.id=6cdd0bbd1eab42f89b3e480afb71beff&scope.project.id=c7512633e55e45dcb163a688cd54156f&role.id=6914eb62761341d7b8640a878225f115 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:04.821468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8bba2-cbed-46d8-888d-0bcc4733d83a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:04.821992 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8bba2-cbed-46d8-888d-0bcc4733d83a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:04.822067 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8bba2-cbed-46d8-888d-0bcc4733d83a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:04.823549 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 101/206] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:04.824509 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c401d-546e-4d1d-85f8-8dc2f89f264c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:04.824899 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c401d-546e-4d1d-85f8-8dc2f89f264c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:04.825079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c401d-546e-4d1d-85f8-8dc2f89f264c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:04.826525 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 106/207] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:04.828189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2d429-14be-4e55-837c-58493a53292b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:04.828476 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2d429-14be-4e55-837c-58493a53292b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:04.828787 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2d429-14be-4e55-837c-58493a53292b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:04.831156 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3667c-c934-4dd5-9564-de13afe7a0ad None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:04.831447 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3667c-c934-4dd5-9564-de13afe7a0ad None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:04.831721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3667c-c934-4dd5-9564-de13afe7a0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:04.848436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e5d2d429-14be-4e55-837c-58493a53292b None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:04.849722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e5d2d429-14be-4e55-837c-58493a53292b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:04.850866 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-67a3667c-c934-4dd5-9564-de13afe7a0ad None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:04.852058 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-67a3667c-c934-4dd5-9564-de13afe7a0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:04.922797 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 107/208] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:04.923654 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 102/209] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:04.971538 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b912858-b361-4627-915f-c2c0a5147926 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:04.971824 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b912858-b361-4627-915f-c2c0a5147926 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:04.972087 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b912858-b361-4627-915f-c2c0a5147926 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:04.973479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 108/210] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:04.978833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d29db-85e5-4d77-b659-512e74aa0b1e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:04.979198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d29db-85e5-4d77-b659-512e74aa0b1e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:04.979551 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d29db-85e5-4d77-b659-512e74aa0b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:04.997722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-573d29db-85e5-4d77-b659-512e74aa0b1e None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:04.998989 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-573d29db-85e5-4d77-b659-512e74aa0b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:05.013187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:05.068256 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 103/211] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:05.189965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b912858-b361-4627-915f-c2c0a5147926 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:05.190955 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d29db-85e5-4d77-b659-512e74aa0b1e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:05.243909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:05.301705 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b912858-b361-4627-915f-c2c0a5147926 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:05.302258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d29db-85e5-4d77-b659-512e74aa0b1e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:05.337592 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57f15b-803f-4e22-91b8-399d966d8f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:05.338323 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57f15b-803f-4e22-91b8-399d966d8f0b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:05.338741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57f15b-803f-4e22-91b8-399d966d8f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:05.339009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57f15b-803f-4e22-91b8-399d966d8f0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:05.340379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c23b87-46b3-4966-bf42-8e212b5dec0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:05.341041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c23b87-46b3-4966-bf42-8e212b5dec0a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:05.341317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c23b87-46b3-4966-bf42-8e212b5dec0a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:05.341588 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c23b87-46b3-4966-bf42-8e212b5dec0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:05.342687 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c23b87-46b3-4966-bf42-8e212b5dec0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:05.344138 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c23b87-46b3-4966-bf42-8e212b5dec0a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:05.345886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 104/212] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:05.349737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57f15b-803f-4e22-91b8-399d966d8f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:05.351424 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c23b87-46b3-4966-bf42-8e212b5dec0a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-3b57f15b-803f-4e22-91b8-399d966d8f0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:05.354553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:05.355586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 109/213] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:05.361371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57f15b-803f-4e22-91b8-399d966d8f0b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:05.369000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:05.450902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c23b87-46b3-4966-bf42-8e212b5dec0a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:05.457879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57f15b-803f-4e22-91b8-399d966d8f0b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:05.468379 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:05.485623 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54273f0-127a-4c2a-ab55-ba1a93a89e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:05.486253 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54273f0-127a-4c2a-ab55-ba1a93a89e77 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:05.486530 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54273f0-127a-4c2a-ab55-ba1a93a89e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:05.486810 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54273f0-127a-4c2a-ab55-ba1a93a89e77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:05.492024 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a3784-af30-4a34-bd53-fa8c0516f44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:05.492646 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a3784-af30-4a34-bd53-fa8c0516f44e None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:05.492927 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a3784-af30-4a34-bd53-fa8c0516f44e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:05.493194 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a3784-af30-4a34-bd53-fa8c0516f44e None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:05.493836 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a3784-af30-4a34-bd53-fa8c0516f44e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:05.495272 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a3784-af30-4a34-bd53-fa8c0516f44e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:05.497547 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54273f0-127a-4c2a-ab55-ba1a93a89e77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a54273f0-127a-4c2a-ab55-ba1a93a89e77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:05.502413 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:05.503324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:05.504104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 105/214] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:05.504813 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54273f0-127a-4c2a-ab55-ba1a93a89e77 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:05.520578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 110/215] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:05 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:05.520578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a3784-af30-4a34-bd53-fa8c0516f44e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:05.594750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:05.622828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54273f0-127a-4c2a-ab55-ba1a93a89e77 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:05.626535 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a3784-af30-4a34-bd53-fa8c0516f44e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:05.668360 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ce3b7-e763-41d0-91c8-63487d1ff330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:05.669034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ce3b7-e763-41d0-91c8-63487d1ff330 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:05.669312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ce3b7-e763-41d0-91c8-63487d1ff330 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:05.669639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ce3b7-e763-41d0-91c8-63487d1ff330 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:05.670284 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ce3b7-e763-41d0-91c8-63487d1ff330 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:05.670585 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ce3b7-e763-41d0-91c8-63487d1ff330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:05.670772 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5afcfb-f603-4707-a24c-32a76cc80035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:05.671404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5afcfb-f603-4707-a24c-32a76cc80035 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:05.671742 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5afcfb-f603-4707-a24c-32a76cc80035 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:05.671970 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ce3b7-e763-41d0-91c8-63487d1ff330 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:05.672062 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5afcfb-f603-4707-a24c-32a76cc80035 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:05.672722 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5afcfb-f603-4707-a24c-32a76cc80035 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:05.673031 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5afcfb-f603-4707-a24c-32a76cc80035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:05.675349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5afcfb-f603-4707-a24c-32a76cc80035 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:05.683244 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 106/216] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:05.683945 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 111/217] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:05.688860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ce3b7-e763-41d0-91c8-63487d1ff330 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:05.689295 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5afcfb-f603-4707-a24c-32a76cc80035 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:05.721942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:05.817391 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5afcfb-f603-4707-a24c-32a76cc80035 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:05.822316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ce3b7-e763-41d0-91c8-63487d1ff330 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:05.864717 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be33f3a-3fb5-4027-b73a-b286813a44ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:05.865371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be33f3a-3fb5-4027-b73a-b286813a44ad None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:05.865654 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be33f3a-3fb5-4027-b73a-b286813a44ad None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:05.865920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be33f3a-3fb5-4027-b73a-b286813a44ad None admin] PATH_INFO: `/v3/users/47165ef79de94355bd56a74009d4f615` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:05.867965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399015f-6ca6-4b0a-87b0-3595578f1db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:05.868650 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399015f-6ca6-4b0a-87b0-3595578f1db9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:05.868941 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399015f-6ca6-4b0a-87b0-3595578f1db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:05.869220 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399015f-6ca6-4b0a-87b0-3595578f1db9 None admin] PATH_INFO: `/v3/users/f760f433bfa14de49fe730653587a515` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:05.870378 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399015f-6ca6-4b0a-87b0-3595578f1db9 None admin] RBAC: Authorizing `identity:get_user(user_id=f760f433bfa14de49fe730653587a515)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:05.871838 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399015f-6ca6-4b0a-87b0-3595578f1db9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:05.873479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 112/218] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:05 2022] GET /identity/v3/users/f760f433bfa14de49fe730653587a515 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:05.877403 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:05.882270 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399015f-6ca6-4b0a-87b0-3595578f1db9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:05.888620 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be33f3a-3fb5-4027-b73a-b286813a44ad None admin] RBAC: Authorizing `identity:get_user(user_id=47165ef79de94355bd56a74009d4f615)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:05.890156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be33f3a-3fb5-4027-b73a-b286813a44ad None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:05.891932 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 107/219] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:05 2022] GET /identity/v3/users/47165ef79de94355bd56a74009d4f615 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:05.901668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be33f3a-3fb5-4027-b73a-b286813a44ad None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:06.007762 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:06.015158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399015f-6ca6-4b0a-87b0-3595578f1db9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:06.022425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be33f3a-3fb5-4027-b73a-b286813a44ad None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:06.054302 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454be043-5b8c-4373-811c-0d7ed26081cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:06.054926 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454be043-5b8c-4373-811c-0d7ed26081cd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:06.055210 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454be043-5b8c-4373-811c-0d7ed26081cd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:06.055483 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454be043-5b8c-4373-811c-0d7ed26081cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:06.062130 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b420889-b33d-4d01-ae86-4fa2d1e4f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:06.063022 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b420889-b33d-4d01-ae86-4fa2d1e4f123 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:06.063516 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b420889-b33d-4d01-ae86-4fa2d1e4f123 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:06.064038 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b420889-b33d-4d01-ae86-4fa2d1e4f123 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:06.065110 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b420889-b33d-4d01-ae86-4fa2d1e4f123 None admin] RBAC: Adding query filter params (role.id=6914eb62761341d7b8640a878225f115, user.id=47165ef79de94355bd56a74009d4f615, scope.system=all) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:06.065666 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b420889-b33d-4d01-ae86-4fa2d1e4f123 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:06.067401 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b420889-b33d-4d01-ae86-4fa2d1e4f123 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:06.073294 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 108/220] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Nov 2 07:14:05 2022] GET /identity/v3/role_assignments?user.id=47165ef79de94355bd56a74009d4f615&scope.system=all&role.id=6914eb62761341d7b8640a878225f115 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:06.134191 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454be043-5b8c-4373-811c-0d7ed26081cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:06.135709 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454be043-5b8c-4373-811c-0d7ed26081cd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:06.248259 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:06.255931 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 113/221] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:14:06.262034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b420889-b33d-4d01-ae86-4fa2d1e4f123 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:06.328157 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:06.372286 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b420889-b33d-4d01-ae86-4fa2d1e4f123 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:06.410145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30135b-a982-44e6-89ce-8c333b3c6a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:06.410794 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30135b-a982-44e6-89ce-8c333b3c6a09 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:06.411074 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30135b-a982-44e6-89ce-8c333b3c6a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:06.411347 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30135b-a982-44e6-89ce-8c333b3c6a09 None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:06.412499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30135b-a982-44e6-89ce-8c333b3c6a09 None admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:06.413981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30135b-a982-44e6-89ce-8c333b3c6a09 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:06.415642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 109/222] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:06 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:06.421545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454be043-5b8c-4373-811c-0d7ed26081cd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:06.530808 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454be043-5b8c-4373-811c-0d7ed26081cd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:06.569983 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5a12f-82c9-4060-8ef3-5f4c98f735f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:06.570984 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5a12f-82c9-4060-8ef3-5f4c98f735f3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:06.571466 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5a12f-82c9-4060-8ef3-5f4c98f735f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:06.571935 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5a12f-82c9-4060-8ef3-5f4c98f735f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:06.572898 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5a12f-82c9-4060-8ef3-5f4c98f735f3 None admin] RBAC: Adding query filter params (role.id=b589362f55584575bde80e8f639f66bb, scope.project.id=c7512633e55e45dcb163a688cd54156f, user.id=f760f433bfa14de49fe730653587a515) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:06.573460 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5a12f-82c9-4060-8ef3-5f4c98f735f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:06.575126 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5a12f-82c9-4060-8ef3-5f4c98f735f3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:06.581876 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 114/223] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:06 2022] GET /identity/v3/role_assignments?user.id=f760f433bfa14de49fe730653587a515&scope.project.id=c7512633e55e45dcb163a688cd54156f&role.id=b589362f55584575bde80e8f639f66bb => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:07.799383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6532a3b-9479-47a3-9385-7c8e12f2eeeb None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:07.800437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6532a3b-9479-47a3-9385-7c8e12f2eeeb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:07.800514 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6532a3b-9479-47a3-9385-7c8e12f2eeeb None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:07.801968 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 110/224] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:07.808069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac257e2-9a3e-4215-a8bf-677e1ce57c71 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:07.808720 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac257e2-9a3e-4215-a8bf-677e1ce57c71 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:07.808720 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac257e2-9a3e-4215-a8bf-677e1ce57c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:07.828042 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5ac257e2-9a3e-4215-a8bf-677e1ce57c71 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:07.829309 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5ac257e2-9a3e-4215-a8bf-677e1ce57c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:07.889190 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:07.901360 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 115/225] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:07.930983 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c518-75c0-4f3f-8ee8-f6cc41b6f9dd None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:07.931293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c518-75c0-4f3f-8ee8-f6cc41b6f9dd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:07.931542 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c518-75c0-4f3f-8ee8-f6cc41b6f9dd None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:07.933071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 111/226] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:07.938293 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf827-f824-43be-b88b-c2f20a4e9de2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:07.938585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf827-f824-43be-b88b-c2f20a4e9de2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:07.938859 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf827-f824-43be-b88b-c2f20a4e9de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:07.958065 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3acbf827-f824-43be-b88b-c2f20a4e9de2 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:07.959240 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3acbf827-f824-43be-b88b-c2f20a4e9de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:08.027459 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 116/227] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:08.172000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0c518-75c0-4f3f-8ee8-f6cc41b6f9dd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:08.277949 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0c518-75c0-4f3f-8ee8-f6cc41b6f9dd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:08.282610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c32d-0c38-4e84-86f0-55b36a15091a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:08.283113 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c32d-0c38-4e84-86f0-55b36a15091a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:08.283673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c32d-0c38-4e84-86f0-55b36a15091a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:08.285765 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 117/228] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:08.291534 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f8cda-4e09-4753-913a-bbb6b71de353 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:08.292071 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f8cda-4e09-4753-913a-bbb6b71de353 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:08.292587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f8cda-4e09-4753-913a-bbb6b71de353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:08.306319 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:08.311406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2f4f8cda-4e09-4753-913a-bbb6b71de353 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:08.312609 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2f4f8cda-4e09-4753-913a-bbb6b71de353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:08.314863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf995c-426a-4260-8701-2f2173c71c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:08.316869 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf995c-426a-4260-8701-2f2173c71c95 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:08.317152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf995c-426a-4260-8701-2f2173c71c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:08.317422 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf995c-426a-4260-8701-2f2173c71c95 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:08.328529 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf995c-426a-4260-8701-2f2173c71c95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-30bf995c-426a-4260-8701-2f2173c71c95 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:08.333406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:08.334330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:08.335126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 112/229] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:08.335787 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf995c-426a-4260-8701-2f2173c71c95 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:08.380735 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 118/230] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:08.381578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f8cda-4e09-4753-913a-bbb6b71de353 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:08.432055 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:08.446786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf995c-426a-4260-8701-2f2173c71c95 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:08.484239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a85a82-6271-47d8-99f4-6a48a9e974ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:08.484905 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a85a82-6271-47d8-99f4-6a48a9e974ba None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:08.485185 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a85a82-6271-47d8-99f4-6a48a9e974ba None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:08.485456 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a85a82-6271-47d8-99f4-6a48a9e974ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:08.486569 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a85a82-6271-47d8-99f4-6a48a9e974ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:08.486926 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f8cda-4e09-4753-913a-bbb6b71de353 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:08.488618 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a85a82-6271-47d8-99f4-6a48a9e974ba None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:08.490193 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 113/231] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:08.495713 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a85a82-6271-47d8-99f4-6a48a9e974ba None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:08.523686 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb320bc-c827-4d67-be88-e57c08a1671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:08.524297 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb320bc-c827-4d67-be88-e57c08a1671b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:08.524710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb320bc-c827-4d67-be88-e57c08a1671b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:08.524982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb320bc-c827-4d67-be88-e57c08a1671b None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:08.525699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb320bc-c827-4d67-be88-e57c08a1671b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:08.526001 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb320bc-c827-4d67-be88-e57c08a1671b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:08.527370 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb320bc-c827-4d67-be88-e57c08a1671b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:08.540572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 119/232] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:08.542247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb320bc-c827-4d67-be88-e57c08a1671b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:08.560168 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:08.606174 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a85a82-6271-47d8-99f4-6a48a9e974ba None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:08.647370 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb320bc-c827-4d67-be88-e57c08a1671b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:08.647742 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ca695-c73e-4a54-9167-1930b1e2bf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:08.648368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ca695-c73e-4a54-9167-1930b1e2bf27 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:08.648674 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ca695-c73e-4a54-9167-1930b1e2bf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:08.649122 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ca695-c73e-4a54-9167-1930b1e2bf27 None admin] PATH_INFO: `/v3/groups` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:08.649758 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ca695-c73e-4a54-9167-1930b1e2bf27 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:08.651184 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ca695-c73e-4a54-9167-1930b1e2bf27 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:08.660984 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:08.667426 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 114/233] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Nov 2 07:14:08 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:14:08.668228 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ca695-c73e-4a54-9167-1930b1e2bf27 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:08.692023 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6097d63-749b-4e82-8081-78be3d2942bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:08.692616 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6097d63-749b-4e82-8081-78be3d2942bb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:08.692902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6097d63-749b-4e82-8081-78be3d2942bb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:08.693172 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6097d63-749b-4e82-8081-78be3d2942bb None admin] PATH_INFO: `/v3/users/87b5ba3da4a74917b9bf981f3120639c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:08.709549 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6097d63-749b-4e82-8081-78be3d2942bb None admin] RBAC: Authorizing `identity:get_user(user_id=87b5ba3da4a74917b9bf981f3120639c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:08.711016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6097d63-749b-4e82-8081-78be3d2942bb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:08.713122 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 120/234] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:08 2022] GET /identity/v3/users/87b5ba3da4a74917b9bf981f3120639c => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:08.721635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6097d63-749b-4e82-8081-78be3d2942bb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:08.788198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ca695-c73e-4a54-9167-1930b1e2bf27 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:08.806516 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:08.825441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d24e-5519-4c6d-a2fe-501044862aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:08.826096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d24e-5519-4c6d-a2fe-501044862aee None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:08.826378 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d24e-5519-4c6d-a2fe-501044862aee None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:08.826650 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d24e-5519-4c6d-a2fe-501044862aee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:08.840022 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12d24e-5519-4c6d-a2fe-501044862aee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:08.841105 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6097d63-749b-4e82-8081-78be3d2942bb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5f12d24e-5519-4c6d-a2fe-501044862aee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:08.845419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:08.847056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:08.847893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 115/235] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:08.853469 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d24e-5519-4c6d-a2fe-501044862aee None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:08.882568 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913da898-446a-483d-bfb0-dbce04675a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:08.883180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913da898-446a-483d-bfb0-dbce04675a48 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:08.883457 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913da898-446a-483d-bfb0-dbce04675a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:08.883726 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913da898-446a-483d-bfb0-dbce04675a48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:08.884507 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913da898-446a-483d-bfb0-dbce04675a48 None admin] RBAC: Adding query filter params (role.id=b589362f55584575bde80e8f639f66bb, user.id=87b5ba3da4a74917b9bf981f3120639c, scope.system=all) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:08.885036 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913da898-446a-483d-bfb0-dbce04675a48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:08.886294 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913da898-446a-483d-bfb0-dbce04675a48 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:08.892879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 121/236] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Nov 2 07:14:08 2022] GET /identity/v3/role_assignments?user.id=87b5ba3da4a74917b9bf981f3120639c&scope.system=all&role.id=b589362f55584575bde80e8f639f66bb => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:08.968598 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d24e-5519-4c6d-a2fe-501044862aee None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:09.006118 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957efe5b-6090-41aa-92c4-a10138db869b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:09.006744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957efe5b-6090-41aa-92c4-a10138db869b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:09.007026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957efe5b-6090-41aa-92c4-a10138db869b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:09.007299 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957efe5b-6090-41aa-92c4-a10138db869b None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:09.007978 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957efe5b-6090-41aa-92c4-a10138db869b None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:09.008284 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957efe5b-6090-41aa-92c4-a10138db869b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:09.009736 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957efe5b-6090-41aa-92c4-a10138db869b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:09.019560 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 116/237] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:09.025317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913da898-446a-483d-bfb0-dbce04675a48 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:09.062160 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:09.131301 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913da898-446a-483d-bfb0-dbce04675a48 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:09.164679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba952aef-8a56-4a27-a792-19101b2ad4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:09.165320 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba952aef-8a56-4a27-a792-19101b2ad4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:09.165600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba952aef-8a56-4a27-a792-19101b2ad4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:09.165872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba952aef-8a56-4a27-a792-19101b2ad4f2 None admin] PATH_INFO: `/v3/users/47165ef79de94355bd56a74009d4f615` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:09.166985 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba952aef-8a56-4a27-a792-19101b2ad4f2 None admin] RBAC: Authorizing `identity:get_user(user_id=47165ef79de94355bd56a74009d4f615)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:09.168429 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba952aef-8a56-4a27-a792-19101b2ad4f2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:09.170209 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 122/238] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:09 2022] GET /identity/v3/users/47165ef79de94355bd56a74009d4f615 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:09.176283 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957efe5b-6090-41aa-92c4-a10138db869b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:09.218845 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:09.280279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957efe5b-6090-41aa-92c4-a10138db869b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:09.314334 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd102b9-360d-4291-84b0-f8cfdbdb827f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:09.314987 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd102b9-360d-4291-84b0-f8cfdbdb827f None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:09.315294 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd102b9-360d-4291-84b0-f8cfdbdb827f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:09.315571 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd102b9-360d-4291-84b0-f8cfdbdb827f None admin] PATH_INFO: `/v3/system/users/47165ef79de94355bd56a74009d4f615/roles/6914eb62761341d7b8640a878225f115` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:09.317002 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd102b9-360d-4291-84b0-f8cfdbdb827f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=47165ef79de94355bd56a74009d4f615, role_id=6914eb62761341d7b8640a878225f115)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:09.318377 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd102b9-360d-4291-84b0-f8cfdbdb827f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:09.326783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 117/239] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Nov 2 07:14:09 2022] PUT /identity/v3/system/users/47165ef79de94355bd56a74009d4f615/roles/6914eb62761341d7b8640a878225f115 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:10.849314 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862a826-b4fd-455b-80a8-178aff7f9e6c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:10.849817 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862a826-b4fd-455b-80a8-178aff7f9e6c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:10.849890 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862a826-b4fd-455b-80a8-178aff7f9e6c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:10.851223 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 123/240] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:10.856331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7d283-bd46-4333-a9d5-bd1f33107dc4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:10.856785 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7d283-bd46-4333-a9d5-bd1f33107dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:10.856946 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7d283-bd46-4333-a9d5-bd1f33107dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:10.874745 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6ed7d283-bd46-4333-a9d5-bd1f33107dc4 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:10.875919 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6ed7d283-bd46-4333-a9d5-bd1f33107dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:10.946220 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 118/241] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:11.041739 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9a6a8-b625-44a7-b7ea-6b3eadb731f7 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:11.042047 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9a6a8-b625-44a7-b7ea-6b3eadb731f7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:11.042301 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9a6a8-b625-44a7-b7ea-6b3eadb731f7 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:11.043782 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 124/242] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:11.048777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9ea66-5991-4cd1-92b0-61bd71e96bf2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:11.049070 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9ea66-5991-4cd1-92b0-61bd71e96bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:11.049348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9ea66-5991-4cd1-92b0-61bd71e96bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:11.067824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-acc9ea66-5991-4cd1-92b0-61bd71e96bf2 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:11.069031 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-acc9ea66-5991-4cd1-92b0-61bd71e96bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:11.094952 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:11.139018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 119/243] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:11.210950 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9a6a8-b625-44a7-b7ea-6b3eadb731f7 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:11.312715 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9a6a8-b625-44a7-b7ea-6b3eadb731f7 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:11.314582 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:11.351802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d984894-6330-406a-a7ec-2ba032502bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:11.352462 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d984894-6330-406a-a7ec-2ba032502bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:11.352791 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d984894-6330-406a-a7ec-2ba032502bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:11.353063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d984894-6330-406a-a7ec-2ba032502bc2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:11.354236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d984894-6330-406a-a7ec-2ba032502bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:11.355738 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d984894-6330-406a-a7ec-2ba032502bc2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:11.357599 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 125/244] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:11.363616 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9ea66-5991-4cd1-92b0-61bd71e96bf2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:11.410509 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d984894-6330-406a-a7ec-2ba032502bc2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:11.469637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9ea66-5991-4cd1-92b0-61bd71e96bf2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:11.503121 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb8ea7-d536-4490-b31e-e47f96ff882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:11.503724 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8ea7-d536-4490-b31e-e47f96ff882b None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:11.504002 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8ea7-d536-4490-b31e-e47f96ff882b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:11.504268 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8ea7-d536-4490-b31e-e47f96ff882b None admin] PATH_INFO: `/v3/groups` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:11.504936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb8ea7-d536-4490-b31e-e47f96ff882b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:11.506353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb8ea7-d536-4490-b31e-e47f96ff882b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:11.510892 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d984894-6330-406a-a7ec-2ba032502bc2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:11.521018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 120/245] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Nov 2 07:14:11 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:14:11.522529 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cba719-e26e-407e-8a15-6d41353cacd1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:11.522818 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cba719-e26e-407e-8a15-6d41353cacd1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:11.523092 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cba719-e26e-407e-8a15-6d41353cacd1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:11.524461 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 121/246] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:11.530275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd87f7-36af-4d57-a076-aee0777a9fba None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:11.530583 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd87f7-36af-4d57-a076-aee0777a9fba None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:11.530837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd87f7-36af-4d57-a076-aee0777a9fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:11.550302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b9cd87f7-36af-4d57-a076-aee0777a9fba None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:11.551542 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b9cd87f7-36af-4d57-a076-aee0777a9fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:11.556872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e4025-1f49-4c1f-818b-2764b3097b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:11.557529 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e4025-1f49-4c1f-818b-2764b3097b30 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:11.557805 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e4025-1f49-4c1f-818b-2764b3097b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:11.558076 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e4025-1f49-4c1f-818b-2764b3097b30 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:11.569636 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e4025-1f49-4c1f-818b-2764b3097b30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:11.570564 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-061e4025-1f49-4c1f-818b-2764b3097b30 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:11.574669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:11.575665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:11.576456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 126/247] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:11.582439 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e4025-1f49-4c1f-818b-2764b3097b30 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:11.615617 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:11.639992 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 122/248] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:11.700916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e4025-1f49-4c1f-818b-2764b3097b30 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:11.737639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bef42-dda9-4f10-9d62-99680991d3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:11.738187 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bef42-dda9-4f10-9d62-99680991d3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:11.738470 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bef42-dda9-4f10-9d62-99680991d3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:11.738747 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bef42-dda9-4f10-9d62-99680991d3f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:11.739417 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bef42-dda9-4f10-9d62-99680991d3f8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:11.739771 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bef42-dda9-4f10-9d62-99680991d3f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:11.741250 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bef42-dda9-4f10-9d62-99680991d3f8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:11.749861 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 127/249] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:11.755924 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd87f7-36af-4d57-a076-aee0777a9fba None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:11.883033 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd87f7-36af-4d57-a076-aee0777a9fba None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:11.910887 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bef42-dda9-4f10-9d62-99680991d3f8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:11.922323 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da2aba-9c47-408e-affa-0ce37aeca582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:11.922974 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2aba-9c47-408e-affa-0ce37aeca582 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:11.923316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2aba-9c47-408e-affa-0ce37aeca582 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:11.923596 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2aba-9c47-408e-affa-0ce37aeca582 None admin] PATH_INFO: `/v3/users/87b5ba3da4a74917b9bf981f3120639c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:11.925439 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da2aba-9c47-408e-affa-0ce37aeca582 None admin] RBAC: Authorizing `identity:get_user(user_id=87b5ba3da4a74917b9bf981f3120639c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:11.926922 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da2aba-9c47-408e-affa-0ce37aeca582 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:11.928627 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 123/250] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:11 2022] GET /identity/v3/users/87b5ba3da4a74917b9bf981f3120639c => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:11.934209 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da2aba-9c47-408e-affa-0ce37aeca582 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:11.956466 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:12.052362 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:12.058890 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da2aba-9c47-408e-affa-0ce37aeca582 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:12.063126 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bef42-dda9-4f10-9d62-99680991d3f8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:12.099991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46567e89-f02b-42e6-9c89-08f91c7b236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:12.101052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46567e89-f02b-42e6-9c89-08f91c7b236f None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:12.101546 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46567e89-f02b-42e6-9c89-08f91c7b236f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:12.102011 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46567e89-f02b-42e6-9c89-08f91c7b236f None admin] PATH_INFO: `/v3/system/users/87b5ba3da4a74917b9bf981f3120639c/roles/b589362f55584575bde80e8f639f66bb` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:12.102658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ae3c8-0b07-4623-9049-91e74713bb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:12.103410 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ae3c8-0b07-4623-9049-91e74713bb2e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:12.103692 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ae3c8-0b07-4623-9049-91e74713bb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:12.103963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ae3c8-0b07-4623-9049-91e74713bb2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:12.104532 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46567e89-f02b-42e6-9c89-08f91c7b236f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=87b5ba3da4a74917b9bf981f3120639c, role_id=b589362f55584575bde80e8f639f66bb)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:12.106264 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46567e89-f02b-42e6-9c89-08f91c7b236f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:12.116502 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ae3c8-0b07-4623-9049-91e74713bb2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:12.117907 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 124/251] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Nov 2 07:14:11 2022] PUT /identity/v3/system/users/87b5ba3da4a74917b9bf981f3120639c/roles/b589362f55584575bde80e8f639f66bb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-052ae3c8-0b07-4623-9049-91e74713bb2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:12.122454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:12.123426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:12.125261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 128/252] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:12.130956 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46567e89-f02b-42e6-9c89-08f91c7b236f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:12.230781 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:12.244631 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46567e89-f02b-42e6-9c89-08f91c7b236f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:12.283097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0511e-7413-4c09-85a7-cafddb703058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:12.283768 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0511e-7413-4c09-85a7-cafddb703058 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:12.284641 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0511e-7413-4c09-85a7-cafddb703058 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:12.284918 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0511e-7413-4c09-85a7-cafddb703058 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:12.285872 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0511e-7413-4c09-85a7-cafddb703058 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:12.286175 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0511e-7413-4c09-85a7-cafddb703058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:12.287776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0511e-7413-4c09-85a7-cafddb703058 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:12.298229 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 125/253] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:12.304216 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ae3c8-0b07-4623-9049-91e74713bb2e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:12.417964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ae3c8-0b07-4623-9049-91e74713bb2e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:12.452620 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b7fbb-0384-4fd5-9a51-d71545418479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:12.453690 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b7fbb-0384-4fd5-9a51-d71545418479 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:12.454265 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b7fbb-0384-4fd5-9a51-d71545418479 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:12.454766 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b7fbb-0384-4fd5-9a51-d71545418479 None admin] PATH_INFO: `/v3/users/47165ef79de94355bd56a74009d4f615` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:12.458496 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b7fbb-0384-4fd5-9a51-d71545418479 None admin] RBAC: Authorizing `identity:get_user(user_id=47165ef79de94355bd56a74009d4f615)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:12.460208 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b7fbb-0384-4fd5-9a51-d71545418479 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:12.462632 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 129/254] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:12 2022] GET /identity/v3/users/47165ef79de94355bd56a74009d4f615 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:12.468242 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0511e-7413-4c09-85a7-cafddb703058 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:12.579856 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0511e-7413-4c09-85a7-cafddb703058 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:12.624899 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a98f25-0926-4342-a1d1-91119a90da62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:12.625895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a98f25-0926-4342-a1d1-91119a90da62 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:12.626367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a98f25-0926-4342-a1d1-91119a90da62 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:12.626846 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a98f25-0926-4342-a1d1-91119a90da62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:12.628306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a98f25-0926-4342-a1d1-91119a90da62 None admin] RBAC: Adding query filter params (role.id=6914eb62761341d7b8640a878225f115, user.id=47165ef79de94355bd56a74009d4f615, scope.system=all) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:12.628845 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a98f25-0926-4342-a1d1-91119a90da62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:12.630622 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a98f25-0926-4342-a1d1-91119a90da62 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:12.649155 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 126/255] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Nov 2 07:14:12 2022] GET /identity/v3/role_assignments?user.id=47165ef79de94355bd56a74009d4f615&scope.system=all&role.id=6914eb62761341d7b8640a878225f115 => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:14.384127 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb25484-efac-4647-93fa-77e325c9d2fd None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:14.385228 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb25484-efac-4647-93fa-77e325c9d2fd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:14.385749 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb25484-efac-4647-93fa-77e325c9d2fd None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:14.387367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 130/256] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:14.394023 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e02231-ee61-4f23-9760-2bb99aa97053 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:14.394986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e02231-ee61-4f23-9760-2bb99aa97053 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:14.395456 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e02231-ee61-4f23-9760-2bb99aa97053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:14.406981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:14.421747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a2e02231-ee61-4f23-9760-2bb99aa97053 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:14.423367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a2e02231-ee61-4f23-9760-2bb99aa97053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:14.502264 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 127/257] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:14:14.652194 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f60a38-b117-4c28-a17a-9a355631b71f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:14.652812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f60a38-b117-4c28-a17a-9a355631b71f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:14.653294 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f60a38-b117-4c28-a17a-9a355631b71f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:14.654831 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 131/258] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:14.660704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80869fda-8d2f-4583-87ab-ed6b88b3f96d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:14.661288 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80869fda-8d2f-4583-87ab-ed6b88b3f96d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:14.661754 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80869fda-8d2f-4583-87ab-ed6b88b3f96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:14.687238 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-80869fda-8d2f-4583-87ab-ed6b88b3f96d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:14.688875 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-80869fda-8d2f-4583-87ab-ed6b88b3f96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:14.703493 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 128/259] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:14:14.805224 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1b5a8-8f6f-41de-8fd9-71a5f95086ae None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:14.805843 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1b5a8-8f6f-41de-8fd9-71a5f95086ae None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:14.806307 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1b5a8-8f6f-41de-8fd9-71a5f95086ae None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:14.808204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 132/260] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:14.822263 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0512b-86e1-4dfc-9ac5-c529c9767fc1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:14.822850 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0512b-86e1-4dfc-9ac5-c529c9767fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:14.823314 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0512b-86e1-4dfc-9ac5-c529c9767fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:14.844476 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7bb0512b-86e1-4dfc-9ac5-c529c9767fc1 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:14.846021 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7bb0512b-86e1-4dfc-9ac5-c529c9767fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:14.854530 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 129/261] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:14:15.050492 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1b5a8-8f6f-41de-8fd9-71a5f95086ae None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.054290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0512b-86e1-4dfc-9ac5-c529c9767fc1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.070804 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1b5a8-8f6f-41de-8fd9-71a5f95086ae None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.077549 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0512b-86e1-4dfc-9ac5-c529c9767fc1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.081291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ca42f-f88a-4d44-b5e6-61cfeb4a8665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.081939 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca42f-f88a-4d44-b5e6-61cfeb4a8665 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.082220 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca42f-f88a-4d44-b5e6-61cfeb4a8665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.082495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca42f-f88a-4d44-b5e6-61cfeb4a8665 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.088072 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dfdd5-dab5-4b3b-80e9-04654543f7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.088734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfdd5-dab5-4b3b-80e9-04654543f7cc admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.089014 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfdd5-dab5-4b3b-80e9-04654543f7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.089285 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfdd5-dab5-4b3b-80e9-04654543f7cc admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.104930 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ca42f-f88a-4d44-b5e6-61cfeb4a8665 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-859ca42f-f88a-4d44-b5e6-61cfeb4a8665 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:15.109834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:15.110679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:15.111479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 133/262] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 62 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:15.121133 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dfdd5-dab5-4b3b-80e9-04654543f7cc admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:15.121425 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ca42f-f88a-4d44-b5e6-61cfeb4a8665 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c27dfdd5-dab5-4b3b-80e9-04654543f7cc admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:15.126990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:15.127921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:15.128887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:15.131482 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 130/263] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 78 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:15.139118 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dfdd5-dab5-4b3b-80e9-04654543f7cc admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.140853 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ca42f-f88a-4d44-b5e6-61cfeb4a8665 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.150750 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea1792-5912-42f9-b0cb-3b59c86c3513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.151553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea1792-5912-42f9-b0cb-3b59c86c3513 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.152102 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea1792-5912-42f9-b0cb-3b59c86c3513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.152658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea1792-5912-42f9-b0cb-3b59c86c3513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.153437 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea1792-5912-42f9-b0cb-3b59c86c3513 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:15.153861 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea1792-5912-42f9-b0cb-3b59c86c3513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.155146 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dfdd5-dab5-4b3b-80e9-04654543f7cc admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.155939 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-eeea1792-5912-42f9-b0cb-3b59c86c3513 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:14:15.155939 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Nov 02 07:14:15.155939 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: Nov 02 07:14:15.158741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea1792-5912-42f9-b0cb-3b59c86c3513 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.162953 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff623a9-55b0-4e05-94a8-9f59c2331b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.163721 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff623a9-55b0-4e05-94a8-9f59c2331b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.164125 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff623a9-55b0-4e05-94a8-9f59c2331b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.164529 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff623a9-55b0-4e05-94a8-9f59c2331b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.165535 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff623a9-55b0-4e05-94a8-9f59c2331b1f admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:15.165934 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:15.166120 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff623a9-55b0-4e05-94a8-9f59c2331b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.167997 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-5ff623a9-55b0-4e05-94a8-9f59c2331b1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:14:15.167997 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Nov 02 07:14:15.167997 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: Nov 02 07:14:15.169275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff623a9-55b0-4e05-94a8-9f59c2331b1f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.179350 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 134/264] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:15.187522 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea1792-5912-42f9-b0cb-3b59c86c3513 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.188935 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 131/265] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:15.201492 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff623a9-55b0-4e05-94a8-9f59c2331b1f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.202493 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea1792-5912-42f9-b0cb-3b59c86c3513 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.218172 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff623a9-55b0-4e05-94a8-9f59c2331b1f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.218762 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733e6cb-c513-43e8-97a6-b8989c3f574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.219570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733e6cb-c513-43e8-97a6-b8989c3f574a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.219951 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733e6cb-c513-43e8-97a6-b8989c3f574a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.220370 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733e6cb-c513-43e8-97a6-b8989c3f574a admin admin] PATH_INFO: `/v3/groups/af5b55cd784f43e2baa3bd1e12783746` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.226544 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f9859-0c38-4795-bf21-d9ad4e2e51be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.227363 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f9859-0c38-4795-bf21-d9ad4e2e51be admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.227780 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f9859-0c38-4795-bf21-d9ad4e2e51be admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.228190 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f9859-0c38-4795-bf21-d9ad4e2e51be admin admin] PATH_INFO: `/v3/groups/af5b55cd784f43e2baa3bd1e12783746` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.233730 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733e6cb-c513-43e8-97a6-b8989c3f574a admin admin] RBAC: Authorizing `identity:get_group(group_id=af5b55cd784f43e2baa3bd1e12783746)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.236639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-3733e6cb-c513-43e8-97a6-b8989c3f574a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:14:15.236639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Nov 02 07:14:15.236639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: Nov 02 07:14:15.236639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733e6cb-c513-43e8-97a6-b8989c3f574a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.239919 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f9859-0c38-4795-bf21-d9ad4e2e51be admin admin] RBAC: Authorizing `identity:get_group(group_id=af5b55cd784f43e2baa3bd1e12783746)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.241159 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-7a3f9859-0c38-4795-bf21-d9ad4e2e51be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:14:15.241159 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Nov 02 07:14:15.241159 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: Nov 02 07:14:15.242612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f9859-0c38-4795-bf21-d9ad4e2e51be admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.244777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 132/267] 10.222.0.15 () {56 vars in 1201 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/groups/af5b55cd784f43e2baa3bd1e12783746 => generated 226 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:15.244890 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 135/267] 10.222.0.15 () {56 vars in 1201 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/groups/af5b55cd784f43e2baa3bd1e12783746 => generated 226 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:15.250660 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f9859-0c38-4795-bf21-d9ad4e2e51be admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.250887 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733e6cb-c513-43e8-97a6-b8989c3f574a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.266627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733e6cb-c513-43e8-97a6-b8989c3f574a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.267714 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f9859-0c38-4795-bf21-d9ad4e2e51be admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.276713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88113409-024d-4c53-909c-76602b050ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.277473 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88113409-024d-4c53-909c-76602b050ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.277840 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d259258-b6a4-47e7-ba0e-96da2c31a1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.277963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88113409-024d-4c53-909c-76602b050ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.278343 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88113409-024d-4c53-909c-76602b050ab5 admin admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.278866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259258-b6a4-47e7-ba0e-96da2c31a1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.279340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259258-b6a4-47e7-ba0e-96da2c31a1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.279732 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88113409-024d-4c53-909c-76602b050ab5 admin admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.279876 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259258-b6a4-47e7-ba0e-96da2c31a1b6 admin admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.281198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d259258-b6a4-47e7-ba0e-96da2c31a1b6 admin admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.282275 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88113409-024d-4c53-909c-76602b050ab5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.283624 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d259258-b6a4-47e7-ba0e-96da2c31a1b6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.285975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 133/268] 10.222.0.15 () {56 vars in 1207 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:15.291865 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 136/269] 10.222.0.15 () {56 vars in 1207 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:15.293083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d259258-b6a4-47e7-ba0e-96da2c31a1b6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.299993 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88113409-024d-4c53-909c-76602b050ab5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.310402 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d259258-b6a4-47e7-ba0e-96da2c31a1b6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.318906 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88113409-024d-4c53-909c-76602b050ab5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.319720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ea09c-e62d-4ac1-8783-3e0e3cd2d1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.320660 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ea09c-e62d-4ac1-8783-3e0e3cd2d1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.321153 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ea09c-e62d-4ac1-8783-3e0e3cd2d1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.321613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ea09c-e62d-4ac1-8783-3e0e3cd2d1cf admin admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47/groups/af5b55cd784f43e2baa3bd1e12783746/roles/1255151437d94364acba841f4215321c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.323996 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ea09c-e62d-4ac1-8783-3e0e3cd2d1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d8e654d0884c519a11a6471e3aee47, group_id=af5b55cd784f43e2baa3bd1e12783746, role_id=1255151437d94364acba841f4215321c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.325274 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-ea2ea09c-e62d-4ac1-8783-3e0e3cd2d1cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:14:15.325274 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Nov 02 07:14:15.325274 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: Nov 02 07:14:15.327691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ea09c-e62d-4ac1-8783-3e0e3cd2d1cf admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.335361 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638e691-1aac-446a-b37b-f20ce1e17a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.336639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e691-1aac-446a-b37b-f20ce1e17a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.337118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e691-1aac-446a-b37b-f20ce1e17a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.337602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638e691-1aac-446a-b37b-f20ce1e17a55 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.353967 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 134/270] 10.222.0.15 () {58 vars in 1463 bytes} [Wed Nov 2 07:14:15 2022] PUT /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47/groups/af5b55cd784f43e2baa3bd1e12783746/roles/1255151437d94364acba841f4215321c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:15.355193 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ea09c-e62d-4ac1-8783-3e0e3cd2d1cf admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.368822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638e691-1aac-446a-b37b-f20ce1e17a55 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2638e691-1aac-446a-b37b-f20ce1e17a55 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:15.374628 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:15.375595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:15.376367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:15.387208 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 137/271] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 88 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:15.393484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638e691-1aac-446a-b37b-f20ce1e17a55 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.417826 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638e691-1aac-446a-b37b-f20ce1e17a55 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.504764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ea09c-e62d-4ac1-8783-3e0e3cd2d1cf admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.515979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f019c8-6951-49fd-a02e-e5da6e3ee68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.517730 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f019c8-6951-49fd-a02e-e5da6e3ee68c admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.518272 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f019c8-6951-49fd-a02e-e5da6e3ee68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.519418 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f019c8-6951-49fd-a02e-e5da6e3ee68c admin admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47/groups/af5b55cd784f43e2baa3bd1e12783746/roles/6914eb62761341d7b8640a878225f115` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.524175 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f019c8-6951-49fd-a02e-e5da6e3ee68c admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d8e654d0884c519a11a6471e3aee47, group_id=af5b55cd784f43e2baa3bd1e12783746, role_id=6914eb62761341d7b8640a878225f115)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.530335 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f019c8-6951-49fd-a02e-e5da6e3ee68c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.538252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851eb429-fda5-43cd-ba51-4b94b6298275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.539222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eb429-fda5-43cd-ba51-4b94b6298275 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.540707 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eb429-fda5-43cd-ba51-4b94b6298275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.541233 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eb429-fda5-43cd-ba51-4b94b6298275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.542148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851eb429-fda5-43cd-ba51-4b94b6298275 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:15.543173 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851eb429-fda5-43cd-ba51-4b94b6298275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.549309 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851eb429-fda5-43cd-ba51-4b94b6298275 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.561821 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 135/272] 10.222.0.15 () {58 vars in 1463 bytes} [Wed Nov 2 07:14:15 2022] PUT /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47/groups/af5b55cd784f43e2baa3bd1e12783746/roles/6914eb62761341d7b8640a878225f115 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:15.566630 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db5f64-7632-4b91-8f54-c21386d2d1c5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.567185 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db5f64-7632-4b91-8f54-c21386d2d1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.567647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db5f64-7632-4b91-8f54-c21386d2d1c5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.567827 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 138/273] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:15.569314 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 136/274] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:15.573873 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851eb429-fda5-43cd-ba51-4b94b6298275 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.582576 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee30a29-f028-466b-ae79-9c8910344b45 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.583184 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee30a29-f028-466b-ae79-9c8910344b45 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.583649 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee30a29-f028-466b-ae79-9c8910344b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.595153 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851eb429-fda5-43cd-ba51-4b94b6298275 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.623141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8ee30a29-f028-466b-ae79-9c8910344b45 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:15.624871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8ee30a29-f028-466b-ae79-9c8910344b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:15.670098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:15.726448 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e9a63-9852-40e2-b2cf-7c0ea84bd6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.727112 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e9a63-9852-40e2-b2cf-7c0ea84bd6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.727397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e9a63-9852-40e2-b2cf-7c0ea84bd6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.727676 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e9a63-9852-40e2-b2cf-7c0ea84bd6ad admin admin] PATH_INFO: `/v3/groups/af5b55cd784f43e2baa3bd1e12783746` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.732759 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e9a63-9852-40e2-b2cf-7c0ea84bd6ad admin admin] RBAC: Authorizing `identity:get_group(group_id=af5b55cd784f43e2baa3bd1e12783746)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.734861 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e9a63-9852-40e2-b2cf-7c0ea84bd6ad admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.737489 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 139/275] 10.222.0.15 () {56 vars in 1201 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/groups/af5b55cd784f43e2baa3bd1e12783746 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:15.742773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e9a63-9852-40e2-b2cf-7c0ea84bd6ad admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.750771 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 137/276] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:15.760783 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e9a63-9852-40e2-b2cf-7c0ea84bd6ad admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.770833 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abef7a-e166-45ce-bde1-3b7a5f9ca6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.771510 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abef7a-e166-45ce-bde1-3b7a5f9ca6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.771825 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abef7a-e166-45ce-bde1-3b7a5f9ca6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.772096 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abef7a-e166-45ce-bde1-3b7a5f9ca6b7 admin admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.773359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abef7a-e166-45ce-bde1-3b7a5f9ca6b7 admin admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.775671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abef7a-e166-45ce-bde1-3b7a5f9ca6b7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.782003 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 140/277] 10.222.0.15 () {56 vars in 1207 bytes} [Wed Nov 2 07:14:15 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:15.783292 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee30a29-f028-466b-ae79-9c8910344b45 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:15.800692 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee30a29-f028-466b-ae79-9c8910344b45 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:15.813465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad5f40-34f9-48d8-a884-25b81d265e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:15.814095 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad5f40-34f9-48d8-a884-25b81d265e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:15.814375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad5f40-34f9-48d8-a884-25b81d265e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:15.814651 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad5f40-34f9-48d8-a884-25b81d265e54 admin admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f/groups/af5b55cd784f43e2baa3bd1e12783746/roles/6914eb62761341d7b8640a878225f115` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:15.816874 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad5f40-34f9-48d8-a884-25b81d265e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7512633e55e45dcb163a688cd54156f, group_id=af5b55cd784f43e2baa3bd1e12783746, role_id=6914eb62761341d7b8640a878225f115)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:15.819543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad5f40-34f9-48d8-a884-25b81d265e54 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:15.838922 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 138/278] 10.222.0.15 () {58 vars in 1463 bytes} [Wed Nov 2 07:14:15 2022] PUT /identity/v3/projects/c7512633e55e45dcb163a688cd54156f/groups/af5b55cd784f43e2baa3bd1e12783746/roles/6914eb62761341d7b8640a878225f115 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:16.100767 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abef7a-e166-45ce-bde1-3b7a5f9ca6b7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:16.170461 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:16.216495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abef7a-e166-45ce-bde1-3b7a5f9ca6b7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:16.258849 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193ba57-67b1-4f98-aa8d-b8424595a1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:16.259493 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193ba57-67b1-4f98-aa8d-b8424595a1be None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:16.259774 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193ba57-67b1-4f98-aa8d-b8424595a1be None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:16.260047 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193ba57-67b1-4f98-aa8d-b8424595a1be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:16.272643 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193ba57-67b1-4f98-aa8d-b8424595a1be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2193ba57-67b1-4f98-aa8d-b8424595a1be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:16.278135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:16.280256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:16.282255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 141/279] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:16.290497 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad5f40-34f9-48d8-a884-25b81d265e54 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:16.421271 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad5f40-34f9-48d8-a884-25b81d265e54 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:16.467503 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329bd3d-5ea9-4154-b02c-c83ffbeb6730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:16.468370 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329bd3d-5ea9-4154-b02c-c83ffbeb6730 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:16.468675 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329bd3d-5ea9-4154-b02c-c83ffbeb6730 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:16.468948 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329bd3d-5ea9-4154-b02c-c83ffbeb6730 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:16.469649 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329bd3d-5ea9-4154-b02c-c83ffbeb6730 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:16.469947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329bd3d-5ea9-4154-b02c-c83ffbeb6730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:16.471892 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329bd3d-5ea9-4154-b02c-c83ffbeb6730 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:16.482269 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 139/280] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:16.489571 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193ba57-67b1-4f98-aa8d-b8424595a1be None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:16.566634 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0151c-282b-449c-af2e-493c3a986c2f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:16.567266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0151c-282b-449c-af2e-493c3a986c2f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:16.567266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0151c-282b-449c-af2e-493c3a986c2f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:16.568738 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 140/281] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:16.574077 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26306dd-626f-4683-8d77-4524261d688d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:16.574399 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26306dd-626f-4683-8d77-4524261d688d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:16.574761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26306dd-626f-4683-8d77-4524261d688d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:16.614057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c26306dd-626f-4683-8d77-4524261d688d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:16.615774 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c26306dd-626f-4683-8d77-4524261d688d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:16.625161 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:16.667677 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193ba57-67b1-4f98-aa8d-b8424595a1be None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:16.689088 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:16.719235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 141/282] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 07:14:16.720784 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6270a3c-bb29-4fb8-8928-101fefac144c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:16.721067 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6270a3c-bb29-4fb8-8928-101fefac144c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:16.721447 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6270a3c-bb29-4fb8-8928-101fefac144c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:16.722286 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3763389-8e6d-46f2-8719-068f64a38167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:16.722831 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 142/283] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:16.724310 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763389-8e6d-46f2-8719-068f64a38167 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:16.724706 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763389-8e6d-46f2-8719-068f64a38167 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:16.725050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763389-8e6d-46f2-8719-068f64a38167 None admin] PATH_INFO: `/v3/users/87b5ba3da4a74917b9bf981f3120639c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:16.729096 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3763389-8e6d-46f2-8719-068f64a38167 None admin] RBAC: Authorizing `identity:get_user(user_id=87b5ba3da4a74917b9bf981f3120639c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:16.730766 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3763389-8e6d-46f2-8719-068f64a38167 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:16.732387 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 142/284] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:16 2022] GET /identity/v3/users/87b5ba3da4a74917b9bf981f3120639c => generated 293 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:16.734062 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c684084-cf2b-47b1-933e-25fd3b14bdd5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:16.734339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c684084-cf2b-47b1-933e-25fd3b14bdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:16.734608 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c684084-cf2b-47b1-933e-25fd3b14bdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:16.739358 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3763389-8e6d-46f2-8719-068f64a38167 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:16.758451 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9c684084-cf2b-47b1-933e-25fd3b14bdd5 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:16.759650 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9c684084-cf2b-47b1-933e-25fd3b14bdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:16.767397 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 143/285] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 07:14:16.891939 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3763389-8e6d-46f2-8719-068f64a38167 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:16.929387 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3476b-1fea-4bae-a21a-4104b776f0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:16.930002 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3476b-1fea-4bae-a21a-4104b776f0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:16.930280 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3476b-1fea-4bae-a21a-4104b776f0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:16.930550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3476b-1fea-4bae-a21a-4104b776f0e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:16.931269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3476b-1fea-4bae-a21a-4104b776f0e1 None admin] RBAC: Adding query filter params (role.id=b589362f55584575bde80e8f639f66bb, user.id=87b5ba3da4a74917b9bf981f3120639c, scope.system=all) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:16.931570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3476b-1fea-4bae-a21a-4104b776f0e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:16.933033 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3476b-1fea-4bae-a21a-4104b776f0e1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:16.947194 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 143/286] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Nov 2 07:14:16 2022] GET /identity/v3/role_assignments?user.id=87b5ba3da4a74917b9bf981f3120639c&scope.system=all&role.id=b589362f55584575bde80e8f639f66bb => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:17.140076 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c684084-cf2b-47b1-933e-25fd3b14bdd5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:17.149985 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c684084-cf2b-47b1-933e-25fd3b14bdd5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:17.157336 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a5c89-9f18-46c8-8abe-ec60e8734122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:17.158042 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5c89-9f18-46c8-8abe-ec60e8734122 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:17.158322 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5c89-9f18-46c8-8abe-ec60e8734122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:17.158588 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5c89-9f18-46c8-8abe-ec60e8734122 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:17.178805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a5c89-9f18-46c8-8abe-ec60e8734122 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b73a5c89-9f18-46c8-8abe-ec60e8734122 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:17.183812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:17.184802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:17.185595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 144/287] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Nov 2 07:14:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:17.191619 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3476b-1fea-4bae-a21a-4104b776f0e1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:17.205818 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3476b-1fea-4bae-a21a-4104b776f0e1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:17.211577 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82356d49-5511-43f2-ba60-7b353fa738e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:17.212198 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82356d49-5511-43f2-ba60-7b353fa738e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:17.212475 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82356d49-5511-43f2-ba60-7b353fa738e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:17.212766 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82356d49-5511-43f2-ba60-7b353fa738e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:17.213438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82356d49-5511-43f2-ba60-7b353fa738e6 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:17.213738 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82356d49-5511-43f2-ba60-7b353fa738e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:17.215445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82356d49-5511-43f2-ba60-7b353fa738e6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:17.227773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 144/288] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Nov 2 07:14:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:17.233465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a5c89-9f18-46c8-8abe-ec60e8734122 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:17.244400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a5c89-9f18-46c8-8abe-ec60e8734122 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:17.252464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83de48-5dca-4801-b5ad-a0606bbf1160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:17.253237 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83de48-5dca-4801-b5ad-a0606bbf1160 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:17.253513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83de48-5dca-4801-b5ad-a0606bbf1160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:17.253785 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83de48-5dca-4801-b5ad-a0606bbf1160 admin admin] PATH_INFO: `/v3/groups/1457fa4026d74370b9cade1ba70537b5` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:17.262540 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83de48-5dca-4801-b5ad-a0606bbf1160 admin admin] RBAC: Authorizing `identity:get_group(group_id=1457fa4026d74370b9cade1ba70537b5)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:17.264623 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83de48-5dca-4801-b5ad-a0606bbf1160 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:17.266482 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 145/289] 10.222.0.15 () {56 vars in 1201 bytes} [Wed Nov 2 07:14:17 2022] GET /identity/v3/groups/1457fa4026d74370b9cade1ba70537b5 => generated 229 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:17.272907 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82356d49-5511-43f2-ba60-7b353fa738e6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:17.284588 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82356d49-5511-43f2-ba60-7b353fa738e6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:17.289965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514831a-d998-4002-91a6-1b152470ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:17.290624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514831a-d998-4002-91a6-1b152470ea15 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:17.290950 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514831a-d998-4002-91a6-1b152470ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:17.291213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514831a-d998-4002-91a6-1b152470ea15 admin admin] PATH_INFO: `/v3/projects/316fe9e17d5440ea82f955c062d5fbe0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:17.292509 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514831a-d998-4002-91a6-1b152470ea15 admin admin] RBAC: Authorizing `identity:get_project(project_id=316fe9e17d5440ea82f955c062d5fbe0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:17.294418 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514831a-d998-4002-91a6-1b152470ea15 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:17.296439 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 145/290] 10.222.0.15 () {56 vars in 1207 bytes} [Wed Nov 2 07:14:17 2022] GET /identity/v3/projects/316fe9e17d5440ea82f955c062d5fbe0 => generated 344 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:17.303114 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83de48-5dca-4801-b5ad-a0606bbf1160 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:17.313878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83de48-5dca-4801-b5ad-a0606bbf1160 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:17.318662 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c6ab6-46ee-4ca5-9c54-8fdd82f43bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:17.319371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c6ab6-46ee-4ca5-9c54-8fdd82f43bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:17.319657 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c6ab6-46ee-4ca5-9c54-8fdd82f43bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:17.319946 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c6ab6-46ee-4ca5-9c54-8fdd82f43bd5 admin admin] PATH_INFO: `/v3/projects/316fe9e17d5440ea82f955c062d5fbe0/groups/1457fa4026d74370b9cade1ba70537b5/roles/81d287a8b606458d87c9607e8b48ac58` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:17.322003 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c6ab6-46ee-4ca5-9c54-8fdd82f43bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=316fe9e17d5440ea82f955c062d5fbe0, group_id=1457fa4026d74370b9cade1ba70537b5, role_id=81d287a8b606458d87c9607e8b48ac58)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:17.324908 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c6ab6-46ee-4ca5-9c54-8fdd82f43bd5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:17.344458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 146/291] 10.222.0.15 () {58 vars in 1463 bytes} [Wed Nov 2 07:14:17 2022] PUT /identity/v3/projects/316fe9e17d5440ea82f955c062d5fbe0/groups/1457fa4026d74370b9cade1ba70537b5/roles/81d287a8b606458d87c9607e8b48ac58 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:17.417438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514831a-d998-4002-91a6-1b152470ea15 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:17.429270 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514831a-d998-4002-91a6-1b152470ea15 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:17.472459 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:17.501883 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b25ebd-242d-4b33-aab2-ea8e6c5dc055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:17.502830 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b25ebd-242d-4b33-aab2-ea8e6c5dc055 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:17.503110 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b25ebd-242d-4b33-aab2-ea8e6c5dc055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:17.503376 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b25ebd-242d-4b33-aab2-ea8e6c5dc055 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:17.519816 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b25ebd-242d-4b33-aab2-ea8e6c5dc055 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-19b25ebd-242d-4b33-aab2-ea8e6c5dc055 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:17.525491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:17.526423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:17.527232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:17.530043 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 146/292] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Nov 2 07:14:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 113 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:17.535647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c6ab6-46ee-4ca5-9c54-8fdd82f43bd5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:17.546919 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c6ab6-46ee-4ca5-9c54-8fdd82f43bd5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:17.552742 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd230d3e-13d0-4a62-b040-df91e3c2a267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:17.553817 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd230d3e-13d0-4a62-b040-df91e3c2a267 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:17.554324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd230d3e-13d0-4a62-b040-df91e3c2a267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:17.554869 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd230d3e-13d0-4a62-b040-df91e3c2a267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:17.555851 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd230d3e-13d0-4a62-b040-df91e3c2a267 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:17.556366 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd230d3e-13d0-4a62-b040-df91e3c2a267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:17.558273 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd230d3e-13d0-4a62-b040-df91e3c2a267 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:17.568193 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 147/293] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Nov 2 07:14:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:17.577686 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b25ebd-242d-4b33-aab2-ea8e6c5dc055 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:17.591777 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b25ebd-242d-4b33-aab2-ea8e6c5dc055 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:17.597892 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b05e5-d2d3-4af8-81b9-1e79a98399c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:17.598831 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b05e5-d2d3-4af8-81b9-1e79a98399c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:17.599363 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b05e5-d2d3-4af8-81b9-1e79a98399c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:17.599857 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b05e5-d2d3-4af8-81b9-1e79a98399c5 admin admin] PATH_INFO: `/v3/groups/af5b55cd784f43e2baa3bd1e12783746` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:17.601348 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b05e5-d2d3-4af8-81b9-1e79a98399c5 admin admin] RBAC: Authorizing `identity:get_group(group_id=af5b55cd784f43e2baa3bd1e12783746)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:17.603534 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b05e5-d2d3-4af8-81b9-1e79a98399c5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:17.605766 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 147/294] 10.222.0.15 () {56 vars in 1201 bytes} [Wed Nov 2 07:14:17 2022] GET /identity/v3/groups/af5b55cd784f43e2baa3bd1e12783746 => generated 226 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:17.610927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd230d3e-13d0-4a62-b040-df91e3c2a267 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:17.623440 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd230d3e-13d0-4a62-b040-df91e3c2a267 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:17.628757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aa03c-1680-4b64-867d-873d1a40f4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:17.629642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa03c-1680-4b64-867d-873d1a40f4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:17.630128 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa03c-1680-4b64-867d-873d1a40f4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:17.630604 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa03c-1680-4b64-867d-873d1a40f4c2 admin admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:17.632042 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60aa03c-1680-4b64-867d-873d1a40f4c2 admin admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:17.634355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60aa03c-1680-4b64-867d-873d1a40f4c2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:17.647959 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 148/295] 10.222.0.15 () {56 vars in 1207 bytes} [Wed Nov 2 07:14:17 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:17.653351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b05e5-d2d3-4af8-81b9-1e79a98399c5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:17.664510 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b05e5-d2d3-4af8-81b9-1e79a98399c5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:17.669381 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d029b-fa72-40b9-af4f-93b3bd0e7114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:17.670290 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d029b-fa72-40b9-af4f-93b3bd0e7114 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:17.670771 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d029b-fa72-40b9-af4f-93b3bd0e7114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:17.671239 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d029b-fa72-40b9-af4f-93b3bd0e7114 admin admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f/groups/af5b55cd784f43e2baa3bd1e12783746/roles/1255151437d94364acba841f4215321c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:17.673659 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d029b-fa72-40b9-af4f-93b3bd0e7114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7512633e55e45dcb163a688cd54156f, group_id=af5b55cd784f43e2baa3bd1e12783746, role_id=1255151437d94364acba841f4215321c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:17.674968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-9a9d029b-fa72-40b9-af4f-93b3bd0e7114 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:14:17.674968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Nov 02 07:14:17.674968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: Nov 02 07:14:17.677304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d029b-fa72-40b9-af4f-93b3bd0e7114 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:17.695156 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 148/296] 10.222.0.15 () {58 vars in 1463 bytes} [Wed Nov 2 07:14:17 2022] PUT /identity/v3/projects/c7512633e55e45dcb163a688cd54156f/groups/af5b55cd784f43e2baa3bd1e12783746/roles/1255151437d94364acba841f4215321c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:18.569628 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4290d1-70ec-4fb6-881b-ba9e541cdf93 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:18.570633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4290d1-70ec-4fb6-881b-ba9e541cdf93 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:18.571120 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4290d1-70ec-4fb6-881b-ba9e541cdf93 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:18.573006 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 149/297] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:18.578173 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c615c7-3ab3-4a10-9e4d-09b31622b727 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:18.579068 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c615c7-3ab3-4a10-9e4d-09b31622b727 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:18.579545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c615c7-3ab3-4a10-9e4d-09b31622b727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:18.603263 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-23c615c7-3ab3-4a10-9e4d-09b31622b727 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:18.604813 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-23c615c7-3ab3-4a10-9e4d-09b31622b727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:18.712134 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 149/298] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:18.741793 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9518b-d150-41f0-b8c7-d8106255ebea None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:18.742099 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9518b-d150-41f0-b8c7-d8106255ebea None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:18.742355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9518b-d150-41f0-b8c7-d8106255ebea None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:18.743844 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 150/299] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:18.749382 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b60a8-6590-42dd-94e5-6bd251158bee None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:18.749906 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b60a8-6590-42dd-94e5-6bd251158bee None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:18.750386 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b60a8-6590-42dd-94e5-6bd251158bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:18.769615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e01b60a8-6590-42dd-94e5-6bd251158bee None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:18.771138 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e01b60a8-6590-42dd-94e5-6bd251158bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:18.847478 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 150/300] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:19.127215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9518b-d150-41f0-b8c7-d8106255ebea None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:19.223805 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b60a8-6590-42dd-94e5-6bd251158bee None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:19.260751 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9518b-d150-41f0-b8c7-d8106255ebea None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:19.264390 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:19.269673 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:19.323538 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56f9b8-78cb-4333-b46e-557ee13e8d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:19.324187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f9b8-78cb-4333-b46e-557ee13e8d24 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:19.324464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f9b8-78cb-4333-b46e-557ee13e8d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:19.324828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f9b8-78cb-4333-b46e-557ee13e8d24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:19.344553 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56f9b8-78cb-4333-b46e-557ee13e8d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ea56f9b8-78cb-4333-b46e-557ee13e8d24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:19.351296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:19.352147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:19.352981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:19.353606 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 151/301] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:14:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:19.364452 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17888184-027b-4aff-b8fb-2ab47e7ab733 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:19.364795 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17888184-027b-4aff-b8fb-2ab47e7ab733 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:19.365080 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17888184-027b-4aff-b8fb-2ab47e7ab733 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:19.366431 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 152/302] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:19.367140 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17888184-027b-4aff-b8fb-2ab47e7ab733 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:19.421308 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b60a8-6590-42dd-94e5-6bd251158bee None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:19.466176 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff528d-4694-4802-ac64-817c22cc2f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:19.466831 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff528d-4694-4802-ac64-817c22cc2f5e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:19.467111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff528d-4694-4802-ac64-817c22cc2f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:19.467383 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff528d-4694-4802-ac64-817c22cc2f5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:19.483150 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff528d-4694-4802-ac64-817c22cc2f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-bbff528d-4694-4802-ac64-817c22cc2f5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:19.490917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:19.491812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:19.492635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:19.493225 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 151/303] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:19.494711 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137c823-9e7e-4b1b-a539-55a0c91ea551 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:19.494711 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137c823-9e7e-4b1b-a539-55a0c91ea551 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:19.494711 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137c823-9e7e-4b1b-a539-55a0c91ea551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:19.504144 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17888184-027b-4aff-b8fb-2ab47e7ab733 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:19.516659 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3137c823-9e7e-4b1b-a539-55a0c91ea551 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:19.520275 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3137c823-9e7e-4b1b-a539-55a0c91ea551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:19.547303 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddbeaa-4488-4b3e-86a2-aaf08aeead85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:19.547930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddbeaa-4488-4b3e-86a2-aaf08aeead85 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:19.548211 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddbeaa-4488-4b3e-86a2-aaf08aeead85 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:19.548485 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddbeaa-4488-4b3e-86a2-aaf08aeead85 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:19.549182 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ddbeaa-4488-4b3e-86a2-aaf08aeead85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:19.550019 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ddbeaa-4488-4b3e-86a2-aaf08aeead85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:19.551665 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ddbeaa-4488-4b3e-86a2-aaf08aeead85 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:19.561516 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 153/304] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:19.562280 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddbeaa-4488-4b3e-86a2-aaf08aeead85 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:19.598075 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:19.611594 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 152/305] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:19.614205 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137c823-9e7e-4b1b-a539-55a0c91ea551 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:19.625646 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:19.700264 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddbeaa-4488-4b3e-86a2-aaf08aeead85 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:19.743730 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896176ac-238b-4088-a14a-2fa068e96307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:19.744887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896176ac-238b-4088-a14a-2fa068e96307 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:19.745373 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896176ac-238b-4088-a14a-2fa068e96307 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:19.745868 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896176ac-238b-4088-a14a-2fa068e96307 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:19.746842 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896176ac-238b-4088-a14a-2fa068e96307 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:19.747346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896176ac-238b-4088-a14a-2fa068e96307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:19.749055 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896176ac-238b-4088-a14a-2fa068e96307 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:19.764283 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 154/306] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:19.771258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896176ac-238b-4088-a14a-2fa068e96307 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:19.789284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137c823-9e7e-4b1b-a539-55a0c91ea551 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:19.832655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da7454-2a85-445f-9085-622ccf815b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:19.833277 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da7454-2a85-445f-9085-622ccf815b0e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:19.833556 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da7454-2a85-445f-9085-622ccf815b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:19.833826 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da7454-2a85-445f-9085-622ccf815b0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:19.887636 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:19.902532 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896176ac-238b-4088-a14a-2fa068e96307 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:19.922380 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da7454-2a85-445f-9085-622ccf815b0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:19.924080 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da7454-2a85-445f-9085-622ccf815b0e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:19.933674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:19.946566 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c453-d673-4ab5-a4eb-00a949f62509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:19.947488 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c453-d673-4ab5-a4eb-00a949f62509 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:19.947985 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c453-d673-4ab5-a4eb-00a949f62509 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:19.948470 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c453-d673-4ab5-a4eb-00a949f62509 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:20.037996 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2c453-d673-4ab5-a4eb-00a949f62509 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:20.039889 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2c453-d673-4ab5-a4eb-00a949f62509 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:20.053567 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 153/307] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:20.059276 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da7454-2a85-445f-9085-622ccf815b0e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:20.151114 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:20.168666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da7454-2a85-445f-9085-622ccf815b0e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:20.172182 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 155/308] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:20.173345 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c453-d673-4ab5-a4eb-00a949f62509 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:20.179528 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:20.210856 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f53c7-f32d-4314-b2fc-479c4f83451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:20.211742 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f53c7-f32d-4314-b2fc-479c4f83451e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:20.212017 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f53c7-f32d-4314-b2fc-479c4f83451e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:20.212286 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f53c7-f32d-4314-b2fc-479c4f83451e None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:20.213436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f53c7-f32d-4314-b2fc-479c4f83451e None admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:20.214879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f53c7-f32d-4314-b2fc-479c4f83451e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:20.218017 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 154/309] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:20 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:20.218871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f53c7-f32d-4314-b2fc-479c4f83451e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:20.306514 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c453-d673-4ab5-a4eb-00a949f62509 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:20.346701 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f53c7-f32d-4314-b2fc-479c4f83451e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:20.347149 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73310cc4-07c8-4804-a77a-6bba5384da83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:20.347935 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73310cc4-07c8-4804-a77a-6bba5384da83 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:20.348038 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73310cc4-07c8-4804-a77a-6bba5384da83 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:20.348309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73310cc4-07c8-4804-a77a-6bba5384da83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:20.361127 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73310cc4-07c8-4804-a77a-6bba5384da83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-73310cc4-07c8-4804-a77a-6bba5384da83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:20.368358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:20.369281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:20.370107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:20.373069 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 156/310] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:20.374182 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73310cc4-07c8-4804-a77a-6bba5384da83 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:20.394700 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648db28-4577-4d7b-89e5-c68cc7b2ff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:20.395826 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648db28-4577-4d7b-89e5-c68cc7b2ff22 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:20.396105 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648db28-4577-4d7b-89e5-c68cc7b2ff22 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:20.396376 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648db28-4577-4d7b-89e5-c68cc7b2ff22 None admin] PATH_INFO: `/v3/projects/77d8e654d0884c519a11a6471e3aee47` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:20.397805 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648db28-4577-4d7b-89e5-c68cc7b2ff22 None admin] RBAC: Authorizing `identity:get_project(project_id=77d8e654d0884c519a11a6471e3aee47)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:20.399513 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648db28-4577-4d7b-89e5-c68cc7b2ff22 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:20.401584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 155/311] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:20 2022] GET /identity/v3/projects/77d8e654d0884c519a11a6471e3aee47 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:20.402580 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648db28-4577-4d7b-89e5-c68cc7b2ff22 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:20.442779 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:20.470264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:20.504304 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73310cc4-07c8-4804-a77a-6bba5384da83 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:20.545707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7c00d-7ac1-401f-a4c1-6051d3de1ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:20.546412 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7c00d-7ac1-401f-a4c1-6051d3de1ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:20.546690 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7c00d-7ac1-401f-a4c1-6051d3de1ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:20.546955 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7c00d-7ac1-401f-a4c1-6051d3de1ba1 None admin] PATH_INFO: `/v3/groups/af5b55cd784f43e2baa3bd1e12783746` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:20.548189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7c00d-7ac1-401f-a4c1-6051d3de1ba1 None admin] RBAC: Authorizing `identity:get_group(group_id=af5b55cd784f43e2baa3bd1e12783746)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:20.550164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7c00d-7ac1-401f-a4c1-6051d3de1ba1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:20.551405 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648db28-4577-4d7b-89e5-c68cc7b2ff22 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:20.551918 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 157/312] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Nov 2 07:14:20 2022] GET /identity/v3/groups/af5b55cd784f43e2baa3bd1e12783746 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:20.552503 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7c00d-7ac1-401f-a4c1-6051d3de1ba1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:20.596981 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5cb7a-e1b3-41bb-b9d5-26417bf3f3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:20.597533 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5cb7a-e1b3-41bb-b9d5-26417bf3f3ae None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:20.597809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5cb7a-e1b3-41bb-b9d5-26417bf3f3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:20.598075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5cb7a-e1b3-41bb-b9d5-26417bf3f3ae None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:20.601311 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5cb7a-e1b3-41bb-b9d5-26417bf3f3ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:20.601933 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5cb7a-e1b3-41bb-b9d5-26417bf3f3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:20.603633 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5cb7a-e1b3-41bb-b9d5-26417bf3f3ae None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:20.616082 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 156/313] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:20.617807 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5cb7a-e1b3-41bb-b9d5-26417bf3f3ae None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:20.681055 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7c00d-7ac1-401f-a4c1-6051d3de1ba1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:20.707974 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:20.732166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758901e8-51bb-4195-ac12-f6a65031e6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:20.732852 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758901e8-51bb-4195-ac12-f6a65031e6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:20.733130 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758901e8-51bb-4195-ac12-f6a65031e6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:20.733401 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758901e8-51bb-4195-ac12-f6a65031e6f0 None admin] PATH_INFO: `/v3/groups/af5b55cd784f43e2baa3bd1e12783746` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:20.734616 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758901e8-51bb-4195-ac12-f6a65031e6f0 None admin] RBAC: Authorizing `identity:get_group(group_id=af5b55cd784f43e2baa3bd1e12783746)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:20.735944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758901e8-51bb-4195-ac12-f6a65031e6f0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:20.738277 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 158/314] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Nov 2 07:14:20 2022] GET /identity/v3/groups/af5b55cd784f43e2baa3bd1e12783746 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:20.739271 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78baca31-5563-4db3-805d-6268cf9f55d9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:20.739554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78baca31-5563-4db3-805d-6268cf9f55d9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:20.739828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78baca31-5563-4db3-805d-6268cf9f55d9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:20.741111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 159/315] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:20.742319 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78baca31-5563-4db3-805d-6268cf9f55d9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:20.760994 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5cb7a-e1b3-41bb-b9d5-26417bf3f3ae None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:20.763320 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:20.804775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2399-3d94-4fdc-b420-540c2c9f998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:20.805390 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2399-3d94-4fdc-b420-540c2c9f998e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:20.805670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2399-3d94-4fdc-b420-540c2c9f998e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:20.805938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2399-3d94-4fdc-b420-540c2c9f998e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:20.806809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2399-3d94-4fdc-b420-540c2c9f998e None admin] RBAC: Adding query filter params (group.id=af5b55cd784f43e2baa3bd1e12783746, role.id=1255151437d94364acba841f4215321c, scope.project.id=77d8e654d0884c519a11a6471e3aee47) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:20.806995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2399-3d94-4fdc-b420-540c2c9f998e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:20.808377 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2399-3d94-4fdc-b420-540c2c9f998e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:20.816021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 157/316] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Nov 2 07:14:20 2022] GET /identity/v3/role_assignments?group.id=af5b55cd784f43e2baa3bd1e12783746&scope.project.id=77d8e654d0884c519a11a6471e3aee47&role.id=1255151437d94364acba841f4215321c => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:20.817056 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbf222-ecec-4e97-b5e6-a07c7cf1c679 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:20.817332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbf222-ecec-4e97-b5e6-a07c7cf1c679 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:20.817605 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbf222-ecec-4e97-b5e6-a07c7cf1c679 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:20.818897 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 158/317] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:20.819463 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbf222-ecec-4e97-b5e6-a07c7cf1c679 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:20.880821 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78baca31-5563-4db3-805d-6268cf9f55d9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:20.924072 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36e224-d6f6-4d84-92a6-acb136cc93a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:20.925165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e224-d6f6-4d84-92a6-acb136cc93a9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:20.925671 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e224-d6f6-4d84-92a6-acb136cc93a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:20.926140 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e224-d6f6-4d84-92a6-acb136cc93a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:20.960455 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbf222-ecec-4e97-b5e6-a07c7cf1c679 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:21.000090 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:21.004834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d5859-2b52-4652-939a-f60bc92e2b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:21.006332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d5859-2b52-4652-939a-f60bc92e2b73 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:21.006616 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d5859-2b52-4652-939a-f60bc92e2b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:21.006887 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d5859-2b52-4652-939a-f60bc92e2b73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:21.007618 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d5859-2b52-4652-939a-f60bc92e2b73 None admin] RBAC: Adding query filter params (group.id=af5b55cd784f43e2baa3bd1e12783746, role.id=6914eb62761341d7b8640a878225f115, scope.project.id=77d8e654d0884c519a11a6471e3aee47) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:21.007927 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d5859-2b52-4652-939a-f60bc92e2b73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:21.009431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d5859-2b52-4652-939a-f60bc92e2b73 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:21.016724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 159/318] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Nov 2 07:14:20 2022] GET /identity/v3/role_assignments?group.id=af5b55cd784f43e2baa3bd1e12783746&scope.project.id=77d8e654d0884c519a11a6471e3aee47&role.id=6914eb62761341d7b8640a878225f115 => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:21.017711 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d4c4d-ab43-4d13-b5ad-e78413f6e81a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:21.017907 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36e224-d6f6-4d84-92a6-acb136cc93a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:21.018005 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d4c4d-ab43-4d13-b5ad-e78413f6e81a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:21.018778 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d4c4d-ab43-4d13-b5ad-e78413f6e81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:21.019871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36e224-d6f6-4d84-92a6-acb136cc93a9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:21.058812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b87d4c4d-ab43-4d13-b5ad-e78413f6e81a None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:21.060384 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b87d4c4d-ab43-4d13-b5ad-e78413f6e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:21.100511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:21.169363 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 160/319] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:21.170669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273e3ad-9214-47d8-969e-015505223660 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:21.170952 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273e3ad-9214-47d8-969e-015505223660 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:21.171227 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273e3ad-9214-47d8-969e-015505223660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:21.180227 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 160/320] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:21.186293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36e224-d6f6-4d84-92a6-acb136cc93a9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:21.194675 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a273e3ad-9214-47d8-969e-015505223660 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:21.196452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a273e3ad-9214-47d8-969e-015505223660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:21.281065 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 161/321] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:21.293999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:21.321623 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36e224-d6f6-4d84-92a6-acb136cc93a9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:21.359421 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654e4b6-4a4a-43f0-86ea-f0a128185afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:21.360053 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654e4b6-4a4a-43f0-86ea-f0a128185afe None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:21.360326 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654e4b6-4a4a-43f0-86ea-f0a128185afe None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:21.360643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654e4b6-4a4a-43f0-86ea-f0a128185afe None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:21.361764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654e4b6-4a4a-43f0-86ea-f0a128185afe None admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:21.363269 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654e4b6-4a4a-43f0-86ea-f0a128185afe None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:21.365283 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 161/322] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:21 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:21.370758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273e3ad-9214-47d8-969e-015505223660 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:21.471131 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654e4b6-4a4a-43f0-86ea-f0a128185afe None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:21.475572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:21.484252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273e3ad-9214-47d8-969e-015505223660 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:21.526195 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0f8cb-b5ef-43a3-8b21-73cb4c8621f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:21.526841 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0f8cb-b5ef-43a3-8b21-73cb4c8621f5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:21.527124 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0f8cb-b5ef-43a3-8b21-73cb4c8621f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:21.527400 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0f8cb-b5ef-43a3-8b21-73cb4c8621f5 None admin] PATH_INFO: `/v3/groups/af5b55cd784f43e2baa3bd1e12783746` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:21.528521 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0f8cb-b5ef-43a3-8b21-73cb4c8621f5 None admin] RBAC: Authorizing `identity:get_group(group_id=af5b55cd784f43e2baa3bd1e12783746)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:21.530025 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0f8cb-b5ef-43a3-8b21-73cb4c8621f5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:21.531895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 162/323] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Nov 2 07:14:21 2022] GET /identity/v3/groups/af5b55cd784f43e2baa3bd1e12783746 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:21.539609 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0f8cb-b5ef-43a3-8b21-73cb4c8621f5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:21.599878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654e4b6-4a4a-43f0-86ea-f0a128185afe None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:21.642505 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd614f-b131-48aa-a6fd-5a294dea0135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:21.643655 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd614f-b131-48aa-a6fd-5a294dea0135 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:21.643655 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd614f-b131-48aa-a6fd-5a294dea0135 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:21.643853 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd614f-b131-48aa-a6fd-5a294dea0135 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:21.656311 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd614f-b131-48aa-a6fd-5a294dea0135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:21.658601 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0f8cb-b5ef-43a3-8b21-73cb4c8621f5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-13dd614f-b131-48aa-a6fd-5a294dea0135 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:21.661681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:21.662574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:21.663441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 162/324] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:21.664086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd614f-b131-48aa-a6fd-5a294dea0135 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:21.680399 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:21.698646 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2092cad-c101-4b45-95ef-a9125511d185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:21.700226 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2092cad-c101-4b45-95ef-a9125511d185 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:21.700881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2092cad-c101-4b45-95ef-a9125511d185 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:21.701439 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2092cad-c101-4b45-95ef-a9125511d185 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:21.702195 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2092cad-c101-4b45-95ef-a9125511d185 None admin] RBAC: Adding query filter params (group.id=af5b55cd784f43e2baa3bd1e12783746, role.id=6914eb62761341d7b8640a878225f115, scope.project.id=c7512633e55e45dcb163a688cd54156f) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:21.702540 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2092cad-c101-4b45-95ef-a9125511d185 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:21.704887 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2092cad-c101-4b45-95ef-a9125511d185 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:21.713174 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 163/325] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Nov 2 07:14:21 2022] GET /identity/v3/role_assignments?group.id=af5b55cd784f43e2baa3bd1e12783746&scope.project.id=c7512633e55e45dcb163a688cd54156f&role.id=6914eb62761341d7b8640a878225f115 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:21.713942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2092cad-c101-4b45-95ef-a9125511d185 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:21.753555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:21.802868 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd614f-b131-48aa-a6fd-5a294dea0135 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:21.856281 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2092cad-c101-4b45-95ef-a9125511d185 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:21.857344 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f2ea7-5a6b-4ed1-b434-a63831674e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:21.858244 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f2ea7-5a6b-4ed1-b434-a63831674e7b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:21.858733 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f2ea7-5a6b-4ed1-b434-a63831674e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:21.859207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f2ea7-5a6b-4ed1-b434-a63831674e7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:21.873909 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f2ea7-5a6b-4ed1-b434-a63831674e7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-671f2ea7-5a6b-4ed1-b434-a63831674e7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:21.879791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:21.880764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:21.881722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 163/326] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:14:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:21.886860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f2ea7-5a6b-4ed1-b434-a63831674e7b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:21.897872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbcf6f7-b9ff-4ace-a67c-dc56227fc9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:21.898490 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcf6f7-b9ff-4ace-a67c-dc56227fc9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:21.898767 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcf6f7-b9ff-4ace-a67c-dc56227fc9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:21.899480 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcf6f7-b9ff-4ace-a67c-dc56227fc9b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:21.899683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbcf6f7-b9ff-4ace-a67c-dc56227fc9b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:21.899982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbcf6f7-b9ff-4ace-a67c-dc56227fc9b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:21.901397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbcf6f7-b9ff-4ace-a67c-dc56227fc9b5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:21.913333 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 164/327] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:21.919591 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbcf6f7-b9ff-4ace-a67c-dc56227fc9b5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:21.992694 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:22.023511 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f2ea7-5a6b-4ed1-b434-a63831674e7b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:22.028026 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:22.045811 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbcf6f7-b9ff-4ace-a67c-dc56227fc9b5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:22.064160 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d1871-e3e0-4f60-958c-b825c47a54e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:22.064888 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d1871-e3e0-4f60-958c-b825c47a54e0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:22.065212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d1871-e3e0-4f60-958c-b825c47a54e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:22.065481 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d1871-e3e0-4f60-958c-b825c47a54e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:22.066143 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d1871-e3e0-4f60-958c-b825c47a54e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:22.066440 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d1871-e3e0-4f60-958c-b825c47a54e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:22.067798 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d1871-e3e0-4f60-958c-b825c47a54e0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:22.076315 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 164/328] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:22.082511 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d1871-e3e0-4f60-958c-b825c47a54e0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:22.083519 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d40625-3cfb-4801-9523-ab04187e38fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:22.084429 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d40625-3cfb-4801-9523-ab04187e38fe None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:22.084741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d40625-3cfb-4801-9523-ab04187e38fe None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:22.085023 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d40625-3cfb-4801-9523-ab04187e38fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:22.170216 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d40625-3cfb-4801-9523-ab04187e38fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:22.172022 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d40625-3cfb-4801-9523-ab04187e38fe None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:22.203488 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d1871-e3e0-4f60-958c-b825c47a54e0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:22.237713 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3019c5f-7b1d-4a04-81e4-17620d53df22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:22.238324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3019c5f-7b1d-4a04-81e4-17620d53df22 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:22.238611 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3019c5f-7b1d-4a04-81e4-17620d53df22 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:22.238888 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3019c5f-7b1d-4a04-81e4-17620d53df22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:22.242246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:22.258327 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:22.273998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 165/329] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:14:22.279467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d40625-3cfb-4801-9523-ab04187e38fe None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:22.307851 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3019c5f-7b1d-4a04-81e4-17620d53df22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:22.309317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3019c5f-7b1d-4a04-81e4-17620d53df22 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:22.375586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d40625-3cfb-4801-9523-ab04187e38fe None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:22.407072 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 165/330] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:14:22.409592 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260d827-af3f-4a00-b881-6c7fe20fa489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:22.410214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260d827-af3f-4a00-b881-6c7fe20fa489 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:22.410489 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260d827-af3f-4a00-b881-6c7fe20fa489 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:22.410758 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260d827-af3f-4a00-b881-6c7fe20fa489 None admin] PATH_INFO: `/v3/projects/316fe9e17d5440ea82f955c062d5fbe0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:22.411866 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260d827-af3f-4a00-b881-6c7fe20fa489 None admin] RBAC: Authorizing `identity:get_project(project_id=316fe9e17d5440ea82f955c062d5fbe0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:22.413239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3019c5f-7b1d-4a04-81e4-17620d53df22 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:22.413346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260d827-af3f-4a00-b881-6c7fe20fa489 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:22.414903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 166/331] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:22 2022] GET /identity/v3/projects/316fe9e17d5440ea82f955c062d5fbe0 => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:22.420051 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260d827-af3f-4a00-b881-6c7fe20fa489 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:22.446036 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:22.475058 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:22.509190 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3019c5f-7b1d-4a04-81e4-17620d53df22 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:22.519241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260d827-af3f-4a00-b881-6c7fe20fa489 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:22.543124 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c80559-c4e4-427c-9830-a27e23a8fb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:22.543680 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c80559-c4e4-427c-9830-a27e23a8fb5e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:22.543960 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c80559-c4e4-427c-9830-a27e23a8fb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:22.544231 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c80559-c4e4-427c-9830-a27e23a8fb5e None admin] PATH_INFO: `/v3/projects/c7512633e55e45dcb163a688cd54156f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:22.545415 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c80559-c4e4-427c-9830-a27e23a8fb5e None admin] RBAC: Authorizing `identity:get_project(project_id=c7512633e55e45dcb163a688cd54156f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:22.546849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c80559-c4e4-427c-9830-a27e23a8fb5e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:22.548513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 166/332] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:22 2022] GET /identity/v3/projects/c7512633e55e45dcb163a688cd54156f => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:22.553183 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ca585-f039-4c58-8d65-6f8a9c99b027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:22.553898 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca585-f039-4c58-8d65-6f8a9c99b027 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:22.554181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca585-f039-4c58-8d65-6f8a9c99b027 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:22.554411 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c80559-c4e4-427c-9830-a27e23a8fb5e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:22.554512 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca585-f039-4c58-8d65-6f8a9c99b027 None admin] PATH_INFO: `/v3/groups/1457fa4026d74370b9cade1ba70537b5` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:22.555523 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ca585-f039-4c58-8d65-6f8a9c99b027 None admin] RBAC: Authorizing `identity:get_group(group_id=1457fa4026d74370b9cade1ba70537b5)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:22.556981 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ca585-f039-4c58-8d65-6f8a9c99b027 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:22.563066 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 167/333] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Nov 2 07:14:22 2022] GET /identity/v3/groups/1457fa4026d74370b9cade1ba70537b5 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:22.564088 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ca585-f039-4c58-8d65-6f8a9c99b027 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:22.651360 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c80559-c4e4-427c-9830-a27e23a8fb5e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:22.653636 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:22.664990 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ca585-f039-4c58-8d65-6f8a9c99b027 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:22.685623 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccda29-1dd3-443c-b911-aa930d2e35ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:22.686259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccda29-1dd3-443c-b911-aa930d2e35ac None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:22.686537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccda29-1dd3-443c-b911-aa930d2e35ac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:22.686807 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccda29-1dd3-443c-b911-aa930d2e35ac None admin] PATH_INFO: `/v3/groups/af5b55cd784f43e2baa3bd1e12783746` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:22.687461 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:22.687913 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccda29-1dd3-443c-b911-aa930d2e35ac None admin] RBAC: Authorizing `identity:get_group(group_id=af5b55cd784f43e2baa3bd1e12783746)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:22.689434 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccda29-1dd3-443c-b911-aa930d2e35ac None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:22.691199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 167/334] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Nov 2 07:14:22 2022] GET /identity/v3/groups/af5b55cd784f43e2baa3bd1e12783746 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:22.696947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccda29-1dd3-443c-b911-aa930d2e35ac None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:22.699203 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f17ae0-2a54-4d4e-9944-c7a3d078603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:22.699811 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17ae0-2a54-4d4e-9944-c7a3d078603f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:22.700093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17ae0-2a54-4d4e-9944-c7a3d078603f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:22.700368 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17ae0-2a54-4d4e-9944-c7a3d078603f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:22.701127 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f17ae0-2a54-4d4e-9944-c7a3d078603f None admin] RBAC: Adding query filter params (group.id=1457fa4026d74370b9cade1ba70537b5, role.id=81d287a8b606458d87c9607e8b48ac58, scope.project.id=316fe9e17d5440ea82f955c062d5fbe0) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:22.701434 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f17ae0-2a54-4d4e-9944-c7a3d078603f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:22.702878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f17ae0-2a54-4d4e-9944-c7a3d078603f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:22.709135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 168/335] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Nov 2 07:14:22 2022] GET /identity/v3/role_assignments?group.id=1457fa4026d74370b9cade1ba70537b5&scope.project.id=316fe9e17d5440ea82f955c062d5fbe0&role.id=81d287a8b606458d87c9607e8b48ac58 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:22.801531 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccda29-1dd3-443c-b911-aa930d2e35ac None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:22.836920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c423a-2687-4c7a-8508-ce9451b6f2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:22.837567 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c423a-2687-4c7a-8508-ce9451b6f2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:22.837806 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c423a-2687-4c7a-8508-ce9451b6f2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:22.838080 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c423a-2687-4c7a-8508-ce9451b6f2e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:22.838811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c423a-2687-4c7a-8508-ce9451b6f2e6 None admin] RBAC: Adding query filter params (group.id=af5b55cd784f43e2baa3bd1e12783746, role.id=1255151437d94364acba841f4215321c, scope.project.id=c7512633e55e45dcb163a688cd54156f) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:22.839124 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c423a-2687-4c7a-8508-ce9451b6f2e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:22.840597 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c423a-2687-4c7a-8508-ce9451b6f2e6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:22.847515 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 168/336] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Nov 2 07:14:22 2022] GET /identity/v3/role_assignments?group.id=af5b55cd784f43e2baa3bd1e12783746&scope.project.id=c7512633e55e45dcb163a688cd54156f&role.id=1255151437d94364acba841f4215321c => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:25.577844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c1858-f7ad-4f79-81d5-cba6a49818b9 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:25.578424 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c1858-f7ad-4f79-81d5-cba6a49818b9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:25.578606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c1858-f7ad-4f79-81d5-cba6a49818b9 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:25.580138 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 169/337] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:25.585890 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed995ae-5e25-4ebd-9321-8f778796fb9d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:25.586312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed995ae-5e25-4ebd-9321-8f778796fb9d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:25.586734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed995ae-5e25-4ebd-9321-8f778796fb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:25.609746 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9ed995ae-5e25-4ebd-9321-8f778796fb9d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:25.610966 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9ed995ae-5e25-4ebd-9321-8f778796fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:25.655487 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:25.707166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 169/338] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:25.947396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cd976-c840-4d9f-aea1-24ae5e75e95a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:25.948141 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cd976-c840-4d9f-aea1-24ae5e75e95a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:25.948671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cd976-c840-4d9f-aea1-24ae5e75e95a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:25.950305 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 170/339] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:25.954824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059693e-13e7-4d3d-a31c-a14dfaa41d7d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:25.955453 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059693e-13e7-4d3d-a31c-a14dfaa41d7d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:25.955983 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059693e-13e7-4d3d-a31c-a14dfaa41d7d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:25.956987 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2622f0c-aac8-4a4b-adfb-f2b23616cb56 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:25.957261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2622f0c-aac8-4a4b-adfb-f2b23616cb56 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:25.957699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2622f0c-aac8-4a4b-adfb-f2b23616cb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:25.958227 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 170/340] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:25.979839 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c9f48-75ca-40d7-b62a-0d885c4c57ba None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:25.979839 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c9f48-75ca-40d7-b62a-0d885c4c57ba None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:25.979839 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c9f48-75ca-40d7-b62a-0d885c4c57ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:25.987626 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b2622f0c-aac8-4a4b-adfb-f2b23616cb56 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:25.988848 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b2622f0c-aac8-4a4b-adfb-f2b23616cb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:26.004267 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bc5c9f48-75ca-40d7-b62a-0d885c4c57ba None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:26.005481 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bc5c9f48-75ca-40d7-b62a-0d885c4c57ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:26.109733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 171/341] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:26.110506 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2622f0c-aac8-4a4b-adfb-f2b23616cb56 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:26.136823 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 171/342] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:26.243203 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2622f0c-aac8-4a4b-adfb-f2b23616cb56 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:26.266457 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:26.293842 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa61cb-c514-4dba-9186-1052e19a39b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:26.294670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa61cb-c514-4dba-9186-1052e19a39b4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:26.294926 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa61cb-c514-4dba-9186-1052e19a39b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:26.295190 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa61cb-c514-4dba-9186-1052e19a39b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:26.307115 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa61cb-c514-4dba-9186-1052e19a39b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:26.308591 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa61cb-c514-4dba-9186-1052e19a39b4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:26.328765 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d7f7d-8a5e-4a87-b9c9-60b7b9279dff None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:26.329047 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d7f7d-8a5e-4a87-b9c9-60b7b9279dff None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:26.329323 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d7f7d-8a5e-4a87-b9c9-60b7b9279dff None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-92fa61cb-c514-4dba-9186-1052e19a39b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:26.329918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:26.330884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:26.331747 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:26.331747 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:26.331747 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:26.331913 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 172/343] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:26.331985 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 172/344] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:26.335786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295a980-3ac2-4f27-ac11-f1989b829786 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:26.336069 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295a980-3ac2-4f27-ac11-f1989b829786 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:26.336895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295a980-3ac2-4f27-ac11-f1989b829786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:26.337391 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa61cb-c514-4dba-9186-1052e19a39b4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:26.364916 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2295a980-3ac2-4f27-ac11-f1989b829786 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:26.366105 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2295a980-3ac2-4f27-ac11-f1989b829786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:26.472157 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa61cb-c514-4dba-9186-1052e19a39b4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:26.477704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 173/345] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:26.478771 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295a980-3ac2-4f27-ac11-f1989b829786 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:26.514687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7dd0d-aa67-4e4e-9385-6d75cbfee385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:26.515230 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7dd0d-aa67-4e4e-9385-6d75cbfee385 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:26.515511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7dd0d-aa67-4e4e-9385-6d75cbfee385 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:26.515784 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7dd0d-aa67-4e4e-9385-6d75cbfee385 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:26.516428 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7dd0d-aa67-4e4e-9385-6d75cbfee385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:26.516755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7dd0d-aa67-4e4e-9385-6d75cbfee385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:26.518168 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7dd0d-aa67-4e4e-9385-6d75cbfee385 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:26.536345 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:26.542226 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 173/346] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:26.548433 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7dd0d-aa67-4e4e-9385-6d75cbfee385 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:26.588426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:26.630298 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295a980-3ac2-4f27-ac11-f1989b829786 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:26.669318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb23da-7e8a-476b-9f3c-75da67191af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:26.669985 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb23da-7e8a-476b-9f3c-75da67191af3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:26.670264 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb23da-7e8a-476b-9f3c-75da67191af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:26.670535 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb23da-7e8a-476b-9f3c-75da67191af3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:26.682723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb23da-7e8a-476b-9f3c-75da67191af3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:26.684148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7dd0d-aa67-4e4e-9385-6d75cbfee385 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:26.684725 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb23da-7e8a-476b-9f3c-75da67191af3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6fcb23da-7e8a-476b-9f3c-75da67191af3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:26.704324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:26.705316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:26.706262 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:26.706262 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:26.706262 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:26.708579 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 174/347] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:26.713898 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb23da-7e8a-476b-9f3c-75da67191af3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:26.728568 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0067c95-c725-4262-a88d-0bad675b158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:26.729700 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0067c95-c725-4262-a88d-0bad675b158b None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:26.730165 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0067c95-c725-4262-a88d-0bad675b158b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:26.730621 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0067c95-c725-4262-a88d-0bad675b158b None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:26.731816 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0067c95-c725-4262-a88d-0bad675b158b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:26.733650 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0067c95-c725-4262-a88d-0bad675b158b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:26.758225 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 174/348] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:26 2022] POST /identity/v3/users => generated 246 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:26.763147 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0067c95-c725-4262-a88d-0bad675b158b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:26.818060 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:26.847491 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb23da-7e8a-476b-9f3c-75da67191af3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:26.888514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:26.893392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07881ec4-0f29-4d74-a366-fe16baf6545b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:26.894041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07881ec4-0f29-4d74-a366-fe16baf6545b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:26.894317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07881ec4-0f29-4d74-a366-fe16baf6545b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:26.894586 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07881ec4-0f29-4d74-a366-fe16baf6545b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:26.912435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07881ec4-0f29-4d74-a366-fe16baf6545b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:26.912677 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0067c95-c725-4262-a88d-0bad675b158b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:26.914653 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07881ec4-0f29-4d74-a366-fe16baf6545b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-07881ec4-0f29-4d74-a366-fe16baf6545b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:26.932356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:26.933343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:26.934168 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:26.934168 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:26.934168 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:26.936495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 175/349] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:26.937758 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07881ec4-0f29-4d74-a366-fe16baf6545b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:26.961142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a15d42-1bfa-449e-bc6f-1a4c51a4a7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:26.962760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a15d42-1bfa-449e-bc6f-1a4c51a4a7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:26.963260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a15d42-1bfa-449e-bc6f-1a4c51a4a7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:26.964033 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a15d42-1bfa-449e-bc6f-1a4c51a4a7e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:26.965235 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a15d42-1bfa-449e-bc6f-1a4c51a4a7e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:26.965726 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a15d42-1bfa-449e-bc6f-1a4c51a4a7e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:26.967425 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a15d42-1bfa-449e-bc6f-1a4c51a4a7e0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:26.981290 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 175/350] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:26.982478 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a15d42-1bfa-449e-bc6f-1a4c51a4a7e0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:27.081061 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07881ec4-0f29-4d74-a366-fe16baf6545b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:27.105565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a15d42-1bfa-449e-bc6f-1a4c51a4a7e0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:27.114596 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:27.121559 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456051d-9dd6-4772-9b0d-30aa6b3767a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:27.122192 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456051d-9dd6-4772-9b0d-30aa6b3767a6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:27.122467 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456051d-9dd6-4772-9b0d-30aa6b3767a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:27.122737 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456051d-9dd6-4772-9b0d-30aa6b3767a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:27.135147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456051d-9dd6-4772-9b0d-30aa6b3767a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:27.137262 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456051d-9dd6-4772-9b0d-30aa6b3767a6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8456051d-9dd6-4772-9b0d-30aa6b3767a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:27.152629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:27.153568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:27.154427 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:27.154427 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:27.154427 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:27.158735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 176/351] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:27.159470 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456051d-9dd6-4772-9b0d-30aa6b3767a6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:27.160012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae902ae-4741-4bf1-a04d-76fea43e7cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:27.160898 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae902ae-4741-4bf1-a04d-76fea43e7cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:27.161399 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae902ae-4741-4bf1-a04d-76fea43e7cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:27.161881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae902ae-4741-4bf1-a04d-76fea43e7cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:27.163074 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae902ae-4741-4bf1-a04d-76fea43e7cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:27.164686 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae902ae-4741-4bf1-a04d-76fea43e7cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:27.166279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae902ae-4741-4bf1-a04d-76fea43e7cf5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:27.182960 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 176/352] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:27.184124 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae902ae-4741-4bf1-a04d-76fea43e7cf5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:27.188054 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:27.273372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456051d-9dd6-4772-9b0d-30aa6b3767a6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:27.296046 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae902ae-4741-4bf1-a04d-76fea43e7cf5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:27.311544 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e612e01-f441-487d-9703-298bda555d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:27.312158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e612e01-f441-487d-9703-298bda555d29 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:27.312440 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e612e01-f441-487d-9703-298bda555d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:27.312737 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e612e01-f441-487d-9703-298bda555d29 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:27.313391 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e612e01-f441-487d-9703-298bda555d29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:27.314830 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e612e01-f441-487d-9703-298bda555d29 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:27.333046 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f63621-7bb9-4773-a604-454192b7a0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:27.333621 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f63621-7bb9-4773-a604-454192b7a0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:27.333904 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f63621-7bb9-4773-a604-454192b7a0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:27.334184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f63621-7bb9-4773-a604-454192b7a0f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:27.334867 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f63621-7bb9-4773-a604-454192b7a0f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:27.335172 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f63621-7bb9-4773-a604-454192b7a0f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:27.337011 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f63621-7bb9-4773-a604-454192b7a0f8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:27.348912 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 177/353] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:27 2022] POST /identity/v3/users => generated 248 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:27.348912 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e612e01-f441-487d-9703-298bda555d29 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:27.355025 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 177/354] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:27.356570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f99852-5d79-4b85-9295-11c39e194d1e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:27.356869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f99852-5d79-4b85-9295-11c39e194d1e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:27.357149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f99852-5d79-4b85-9295-11c39e194d1e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:27.358531 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 178/355] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:27.359355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f99852-5d79-4b85-9295-11c39e194d1e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:27.405393 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:27.438031 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:27.484788 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f99852-5d79-4b85-9295-11c39e194d1e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:27.507597 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e612e01-f441-487d-9703-298bda555d29 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:27.526823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb81b03-2be4-40dd-960c-92da2735dd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:27.527453 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb81b03-2be4-40dd-960c-92da2735dd50 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:27.527784 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb81b03-2be4-40dd-960c-92da2735dd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:27.528067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb81b03-2be4-40dd-960c-92da2735dd50 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:27.528763 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb81b03-2be4-40dd-960c-92da2735dd50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:27.530207 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb81b03-2be4-40dd-960c-92da2735dd50 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:27.558115 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 179/356] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:27 2022] POST /identity/v3/users => generated 249 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:27.562755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd883ca-7995-44c1-8b7d-3c56850189eb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:27.562755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd883ca-7995-44c1-8b7d-3c56850189eb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:27.562755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd883ca-7995-44c1-8b7d-3c56850189eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:27.566349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b21988-8916-454c-a61e-3ba3c7bf5bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:27.566992 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b21988-8916-454c-a61e-3ba3c7bf5bfb None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:27.567321 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b21988-8916-454c-a61e-3ba3c7bf5bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:27.567771 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b21988-8916-454c-a61e-3ba3c7bf5bfb None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:27.568470 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b21988-8916-454c-a61e-3ba3c7bf5bfb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:27.570034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b21988-8916-454c-a61e-3ba3c7bf5bfb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:27.589037 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8cd883ca-7995-44c1-8b7d-3c56850189eb None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:27.591016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8cd883ca-7995-44c1-8b7d-3c56850189eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:27.611222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 178/357] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:27 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:27.688834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 180/358] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:27.967410 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b21988-8916-454c-a61e-3ba3c7bf5bfb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:28.071173 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:28.084980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b21988-8916-454c-a61e-3ba3c7bf5bfb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:28.132703 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c0673-79b3-40d2-86ab-39aff59eee26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:28.133324 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c0673-79b3-40d2-86ab-39aff59eee26 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:28.133604 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c0673-79b3-40d2-86ab-39aff59eee26 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:28.133877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c0673-79b3-40d2-86ab-39aff59eee26 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:28.134554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c0673-79b3-40d2-86ab-39aff59eee26 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:28.135976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c0673-79b3-40d2-86ab-39aff59eee26 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:28.151847 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.sql.core [None req-d99c0673-79b3-40d2-86ab-39aff59eee26 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:14:28.151847 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 07:14:28.151847 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [parameters: {'id': 'e7cd2dcd585e4852ba35cc3ae51f7603', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 07:14:28.151847 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81400) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d99c0673-79b3-40d2-86ab-39aff59eee26 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Nov 02 07:14:28.159881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return ref.to_dict() Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:14:28.160940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application yield resource Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise self.value Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application session.commit() Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:14:28.162071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:14:28.163258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:14:28.165374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:14:28.167306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [parameters: {'id': 'e7cd2dcd585e4852ba35cc3ae51f7603', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:28.169265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 07:14:28.171226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:28.177684 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 179/359] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Nov 2 07:14:27 2022] POST /identity/v3/roles => generated 144 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:28.177831 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd883ca-7995-44c1-8b7d-3c56850189eb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:28.255410 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:28.319537 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd883ca-7995-44c1-8b7d-3c56850189eb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:28.357664 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a0f20-59a7-4fca-b991-c72f5cb13842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:28.358304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a0f20-59a7-4fca-b991-c72f5cb13842 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:28.358585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a0f20-59a7-4fca-b991-c72f5cb13842 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:28.358860 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a0f20-59a7-4fca-b991-c72f5cb13842 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:28.369530 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a0f20-59a7-4fca-b991-c72f5cb13842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0b2a0f20-59a7-4fca-b991-c72f5cb13842 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:28.374613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:28.376669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:28.379247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 181/360] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:14:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:28.383227 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c0673-79b3-40d2-86ab-39aff59eee26 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:28.493751 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c0673-79b3-40d2-86ab-39aff59eee26 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:28.541438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489460e9-32c4-4464-872e-d13c853f4974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:28.541977 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489460e9-32c4-4464-872e-d13c853f4974 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:28.542259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489460e9-32c4-4464-872e-d13c853f4974 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:28.542535 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489460e9-32c4-4464-872e-d13c853f4974 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:28.543210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489460e9-32c4-4464-872e-d13c853f4974 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:28.543516 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489460e9-32c4-4464-872e-d13c853f4974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:28.544911 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489460e9-32c4-4464-872e-d13c853f4974 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:28.556557 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 180/361] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:29.210293 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ed618-b53c-4609-ad21-4504c5f6bea1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:29.210831 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ed618-b53c-4609-ad21-4504c5f6bea1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:29.210831 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ed618-b53c-4609-ad21-4504c5f6bea1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:29.212740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 182/362] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:29.217949 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7e75f-6dd0-4183-a4a4-b3bb60660ef2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:29.218346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7e75f-6dd0-4183-a4a4-b3bb60660ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:29.218521 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7e75f-6dd0-4183-a4a4-b3bb60660ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:29.237201 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3dd7e75f-6dd0-4183-a4a4-b3bb60660ef2 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:29.238465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3dd7e75f-6dd0-4183-a4a4-b3bb60660ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:29.252723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:29.330284 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 181/363] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:29.750299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ed618-b53c-4609-ad21-4504c5f6bea1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:29.886151 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ed618-b53c-4609-ad21-4504c5f6bea1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:29.897753 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:29.930615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99682e43-18c7-45a6-aa12-817ba56bdc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:29.931609 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99682e43-18c7-45a6-aa12-817ba56bdc1a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:29.932125 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99682e43-18c7-45a6-aa12-817ba56bdc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:29.932617 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99682e43-18c7-45a6-aa12-817ba56bdc1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:29.949238 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99682e43-18c7-45a6-aa12-817ba56bdc1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-99682e43-18c7-45a6-aa12-817ba56bdc1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:29.954516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:29.955385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:29.956755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:29.959055 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 183/364] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:29.965206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7e75f-6dd0-4183-a4a4-b3bb60660ef2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:30.096343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7e75f-6dd0-4183-a4a4-b3bb60660ef2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:30.145515 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b36618-eb3b-41cf-9ee6-a480711d41cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:30.146542 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b36618-eb3b-41cf-9ee6-a480711d41cd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.147068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b36618-eb3b-41cf-9ee6-a480711d41cd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.147541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b36618-eb3b-41cf-9ee6-a480711d41cd None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.148519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b36618-eb3b-41cf-9ee6-a480711d41cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:30.149063 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b36618-eb3b-41cf-9ee6-a480711d41cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:30.150707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b36618-eb3b-41cf-9ee6-a480711d41cd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:30.160900 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 182/365] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:30.166644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99682e43-18c7-45a6-aa12-817ba56bdc1a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:30.264608 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20632d0-336b-402b-b141-9f205adecc1a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.265150 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20632d0-336b-402b-b141-9f205adecc1a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.265150 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20632d0-336b-402b-b141-9f205adecc1a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.266780 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 183/366] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:30.276564 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335b704-2299-41e4-a62f-7f584cda5569 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.276564 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335b704-2299-41e4-a62f-7f584cda5569 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.276564 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335b704-2299-41e4-a62f-7f584cda5569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.306367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e335b704-2299-41e4-a62f-7f584cda5569 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:30.308041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e335b704-2299-41e4-a62f-7f584cda5569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:30.316510 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:30.326783 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99682e43-18c7-45a6-aa12-817ba56bdc1a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:30.364568 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db815a4-656a-4eee-a958-a6179bc24fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:30.365237 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db815a4-656a-4eee-a958-a6179bc24fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.365518 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db815a4-656a-4eee-a958-a6179bc24fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.365795 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db815a4-656a-4eee-a958-a6179bc24fe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.376649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db815a4-656a-4eee-a958-a6179bc24fe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:30.378639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db815a4-656a-4eee-a958-a6179bc24fe4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1db815a4-656a-4eee-a958-a6179bc24fe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:30.393060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:30.394021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:30.394875 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:30.394875 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:30.394875 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:30.394875 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 184/367] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:30.399922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac35df-2b9d-4b61-90f3-e8b7dbc2b623 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.399922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac35df-2b9d-4b61-90f3-e8b7dbc2b623 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.399922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac35df-2b9d-4b61-90f3-e8b7dbc2b623 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.401604 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 185/368] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:30.402386 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 184/369] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:30.402844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac35df-2b9d-4b61-90f3-e8b7dbc2b623 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:30.405007 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3431b-1cef-4f06-9999-c30f9618a756 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.405652 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3431b-1cef-4f06-9999-c30f9618a756 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.406128 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3431b-1cef-4f06-9999-c30f9618a756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.426946 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-dfd3431b-1cef-4f06-9999-c30f9618a756 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:30.428142 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-dfd3431b-1cef-4f06-9999-c30f9618a756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:30.433565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:30.511323 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 185/370] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:30.513053 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa08a8a-ed89-476a-8ffd-7932e55df568 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.513447 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa08a8a-ed89-476a-8ffd-7932e55df568 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.513726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa08a8a-ed89-476a-8ffd-7932e55df568 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.515084 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 186/371] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:30.520904 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5e66f-c21b-4793-a38b-bcbd361b0666 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.521319 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5e66f-c21b-4793-a38b-bcbd361b0666 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.521736 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5e66f-c21b-4793-a38b-bcbd361b0666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.527216 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac35df-2b9d-4b61-90f3-e8b7dbc2b623 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:30.543723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c3e5e66f-c21b-4793-a38b-bcbd361b0666 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:30.545101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c3e5e66f-c21b-4793-a38b-bcbd361b0666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:30.593237 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f73126-6aeb-4897-8e37-d89c06aad9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:30.593863 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f73126-6aeb-4897-8e37-d89c06aad9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.594148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f73126-6aeb-4897-8e37-d89c06aad9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.594422 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f73126-6aeb-4897-8e37-d89c06aad9d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.595071 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f73126-6aeb-4897-8e37-d89c06aad9d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:30.595372 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f73126-6aeb-4897-8e37-d89c06aad9d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:30.597283 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f73126-6aeb-4897-8e37-d89c06aad9d8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:30.627104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 186/373] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:30.627344 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 187/373] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:30.632676 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f73126-6aeb-4897-8e37-d89c06aad9d8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:30.699895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5e66f-c21b-4793-a38b-bcbd361b0666 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:30.702883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:30.777572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:30.794836 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f73126-6aeb-4897-8e37-d89c06aad9d8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:30.832792 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258a8d9-640b-436e-ae52-02db67d85140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:30.833424 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a8d9-640b-436e-ae52-02db67d85140 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.833698 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a8d9-640b-436e-ae52-02db67d85140 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.833967 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a8d9-640b-436e-ae52-02db67d85140 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.841678 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258a8d9-640b-436e-ae52-02db67d85140 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:30.843101 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258a8d9-640b-436e-ae52-02db67d85140 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d258a8d9-640b-436e-ae52-02db67d85140 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:30.858284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:30.859260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:30.860148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:30.860148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:30.860148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:30.860148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:30.860148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:30.860148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:30.860148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:30.860148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:30.860148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:14:30.860148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:30.860148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 187/374] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Nov 2 07:14:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:30.865453 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258a8d9-640b-436e-ae52-02db67d85140 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:30.893464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5e66f-c21b-4793-a38b-bcbd361b0666 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:30.937660 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cddc08-45ea-446a-b6a9-70dafe57b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:30.938294 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cddc08-45ea-446a-b6a9-70dafe57b914 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.938570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cddc08-45ea-446a-b6a9-70dafe57b914 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.938839 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cddc08-45ea-446a-b6a9-70dafe57b914 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.949771 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cddc08-45ea-446a-b6a9-70dafe57b914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-74cddc08-45ea-446a-b6a9-70dafe57b914 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:30.954589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:30.955513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:30.956306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:30.960369 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 188/375] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:30.961103 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cddc08-45ea-446a-b6a9-70dafe57b914 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:30.975066 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258a8d9-640b-436e-ae52-02db67d85140 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.012697 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f975789-ce72-4f78-b218-54368bcf9b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.013329 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f975789-ce72-4f78-b218-54368bcf9b23 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.013601 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f975789-ce72-4f78-b218-54368bcf9b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.013867 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f975789-ce72-4f78-b218-54368bcf9b23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.023955 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:31.024709 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f975789-ce72-4f78-b218-54368bcf9b23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6f975789-ce72-4f78-b218-54368bcf9b23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:31.030541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:31.031458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:31.032264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 188/376] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:31.033498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f975789-ce72-4f78-b218-54368bcf9b23 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.035417 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:31.073277 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cddc08-45ea-446a-b6a9-70dafe57b914 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.114155 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27c03d-f065-4711-a9e0-c64e376c3773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.115197 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27c03d-f065-4711-a9e0-c64e376c3773 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.115484 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27c03d-f065-4711-a9e0-c64e376c3773 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.115751 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27c03d-f065-4711-a9e0-c64e376c3773 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.116489 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27c03d-f065-4711-a9e0-c64e376c3773 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:31.117178 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27c03d-f065-4711-a9e0-c64e376c3773 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.119215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27c03d-f065-4711-a9e0-c64e376c3773 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.139786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f975789-ce72-4f78-b218-54368bcf9b23 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.162562 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 189/377] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Nov 2 07:14:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:31.163596 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27c03d-f065-4711-a9e0-c64e376c3773 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.177995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0548d-39ad-4989-a5af-74aaee3c8015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.178639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0548d-39ad-4989-a5af-74aaee3c8015 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.178922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0548d-39ad-4989-a5af-74aaee3c8015 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.179197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0548d-39ad-4989-a5af-74aaee3c8015 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.189599 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0548d-39ad-4989-a5af-74aaee3c8015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-93b0548d-39ad-4989-a5af-74aaee3c8015 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:31.195564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:31.196474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 189/378] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:31.202359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0548d-39ad-4989-a5af-74aaee3c8015 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.266186 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:31.275223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27c03d-f065-4711-a9e0-c64e376c3773 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.299419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:31.308530 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0548d-39ad-4989-a5af-74aaee3c8015 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.314290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c432db-39fd-4b3f-b0fb-b5515ed2ed89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.315425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c432db-39fd-4b3f-b0fb-b5515ed2ed89 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.315702 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c432db-39fd-4b3f-b0fb-b5515ed2ed89 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.315972 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c432db-39fd-4b3f-b0fb-b5515ed2ed89 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.316707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c432db-39fd-4b3f-b0fb-b5515ed2ed89 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:31.317014 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c432db-39fd-4b3f-b0fb-b5515ed2ed89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.318396 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c432db-39fd-4b3f-b0fb-b5515ed2ed89 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.331875 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 190/379] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:31.331875 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c432db-39fd-4b3f-b0fb-b5515ed2ed89 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.344149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3dd58-f199-4102-83ef-1060b7750cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.345013 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3dd58-f199-4102-83ef-1060b7750cad None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.345124 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3dd58-f199-4102-83ef-1060b7750cad None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.345324 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3dd58-f199-4102-83ef-1060b7750cad None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.345994 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3dd58-f199-4102-83ef-1060b7750cad None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:31.346800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3dd58-f199-4102-83ef-1060b7750cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.348241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3dd58-f199-4102-83ef-1060b7750cad None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.360569 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 190/380] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:31.360569 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3dd58-f199-4102-83ef-1060b7750cad None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.449736 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c432db-39fd-4b3f-b0fb-b5515ed2ed89 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.468504 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3dd58-f199-4102-83ef-1060b7750cad None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.486373 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12080873-9a03-4d7c-86a2-ef5cd16a6645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.487112 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12080873-9a03-4d7c-86a2-ef5cd16a6645 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.487391 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12080873-9a03-4d7c-86a2-ef5cd16a6645 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.487664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12080873-9a03-4d7c-86a2-ef5cd16a6645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.492364 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:31.499574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12080873-9a03-4d7c-86a2-ef5cd16a6645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.501055 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12080873-9a03-4d7c-86a2-ef5cd16a6645 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.505077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371cf7a-ab87-448d-aa39-963fa6185716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.505699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371cf7a-ab87-448d-aa39-963fa6185716 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.505976 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371cf7a-ab87-448d-aa39-963fa6185716 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.506247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371cf7a-ab87-448d-aa39-963fa6185716 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.506899 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371cf7a-ab87-448d-aa39-963fa6185716 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:31.507199 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371cf7a-ab87-448d-aa39-963fa6185716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.509017 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371cf7a-ab87-448d-aa39-963fa6185716 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-12080873-9a03-4d7c-86a2-ef5cd16a6645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:31.516457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:31.517362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:31.518787 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:31.518787 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:31.518787 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:31.518959 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 191/381] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:31.519028 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 191/382] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:31.519101 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371cf7a-ab87-448d-aa39-963fa6185716 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.519175 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12080873-9a03-4d7c-86a2-ef5cd16a6645 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.562648 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:31.625318 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371cf7a-ab87-448d-aa39-963fa6185716 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.628495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12080873-9a03-4d7c-86a2-ef5cd16a6645 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.658326 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3b07e-5481-426a-9fe1-44ff7c8f8795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.659513 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3b07e-5481-426a-9fe1-44ff7c8f8795 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.659791 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3b07e-5481-426a-9fe1-44ff7c8f8795 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.660059 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3b07e-5481-426a-9fe1-44ff7c8f8795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.665087 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e00e89-fcda-49a8-89d0-1645b7b2e47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.665709 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e00e89-fcda-49a8-89d0-1645b7b2e47f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.665992 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e00e89-fcda-49a8-89d0-1645b7b2e47f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.666259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e00e89-fcda-49a8-89d0-1645b7b2e47f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.670340 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3b07e-5481-426a-9fe1-44ff7c8f8795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.671778 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3b07e-5481-426a-9fe1-44ff7c8f8795 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.676604 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e00e89-fcda-49a8-89d0-1645b7b2e47f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.678026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e00e89-fcda-49a8-89d0-1645b7b2e47f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-edd3b07e-5481-426a-9fe1-44ff7c8f8795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:31.685674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:31.686704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:31.687714 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:31.687714 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:31.687714 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:31.687714 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 192/383] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:31.687714 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3b07e-5481-426a-9fe1-44ff7c8f8795 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-80e00e89-fcda-49a8-89d0-1645b7b2e47f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:31.696356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:31.697302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:31.698143 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:31.698143 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:31.698143 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:31.698143 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 192/384] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:31.699669 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e00e89-fcda-49a8-89d0-1645b7b2e47f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.727042 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:31.784102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:31.785842 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3b07e-5481-426a-9fe1-44ff7c8f8795 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.800113 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e00e89-fcda-49a8-89d0-1645b7b2e47f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.818263 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f477b3-0596-4d35-9f61-13f91b7b9f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.818893 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f477b3-0596-4d35-9f61-13f91b7b9f45 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.819171 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f477b3-0596-4d35-9f61-13f91b7b9f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.819442 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f477b3-0596-4d35-9f61-13f91b7b9f45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.830099 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f477b3-0596-4d35-9f61-13f91b7b9f45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.832038 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f477b3-0596-4d35-9f61-13f91b7b9f45 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.832812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e021a-174c-48fb-817a-03e08d2f6844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.833405 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e021a-174c-48fb-817a-03e08d2f6844 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.833716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e021a-174c-48fb-817a-03e08d2f6844 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.833986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e021a-174c-48fb-817a-03e08d2f6844 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.834616 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e021a-174c-48fb-817a-03e08d2f6844 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:31.834912 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e021a-174c-48fb-817a-03e08d2f6844 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.836248 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e021a-174c-48fb-817a-03e08d2f6844 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-65f477b3-0596-4d35-9f61-13f91b7b9f45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:31.846197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:31.847287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:31.848125 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:31.848125 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:31.848125 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:31.848125 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 193/385] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:31.848346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 193/386] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:31.848412 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302769-b2f9-40c7-9ff7-713adac428e9 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.848850 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302769-b2f9-40c7-9ff7-713adac428e9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.848946 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e021a-174c-48fb-817a-03e08d2f6844 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.849029 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302769-b2f9-40c7-9ff7-713adac428e9 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.850290 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 194/387] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:31.850808 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc302769-b2f9-40c7-9ff7-713adac428e9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.943170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:31.951195 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e021a-174c-48fb-817a-03e08d2f6844 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.953877 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc302769-b2f9-40c7-9ff7-713adac428e9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.985014 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80774e29-f620-4c27-a5ea-7644481fd0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.985614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80774e29-f620-4c27-a5ea-7644481fd0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.985891 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80774e29-f620-4c27-a5ea-7644481fd0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.986164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80774e29-f620-4c27-a5ea-7644481fd0e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.986600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c78dc5-3eda-4952-96c8-ef944136418c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.986844 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80774e29-f620-4c27-a5ea-7644481fd0e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:31.987144 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80774e29-f620-4c27-a5ea-7644481fd0e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.987230 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78dc5-3eda-4952-96c8-ef944136418c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.987501 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78dc5-3eda-4952-96c8-ef944136418c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.987768 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78dc5-3eda-4952-96c8-ef944136418c None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.988391 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c78dc5-3eda-4952-96c8-ef944136418c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:31.988502 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80774e29-f620-4c27-a5ea-7644481fd0e0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.988717 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c78dc5-3eda-4952-96c8-ef944136418c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.990049 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c78dc5-3eda-4952-96c8-ef944136418c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.999416 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 194/388] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:32.000197 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80774e29-f620-4c27-a5ea-7644481fd0e0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.004873 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 195/389] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:32.004873 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca65067-f3c7-4192-bc73-acf4ab28b322 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.004873 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca65067-f3c7-4192-bc73-acf4ab28b322 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.004873 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca65067-f3c7-4192-bc73-acf4ab28b322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.011563 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:32.021630 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-cca65067-f3c7-4192-bc73-acf4ab28b322 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:32.022815 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cca65067-f3c7-4192-bc73-acf4ab28b322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:32.087421 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 196/390] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:32.088167 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca65067-f3c7-4192-bc73-acf4ab28b322 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.105418 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80774e29-f620-4c27-a5ea-7644481fd0e0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.138633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14f974-ec1c-415e-8765-a8d5c16ff33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.139240 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f974-ec1c-415e-8765-a8d5c16ff33a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.139521 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f974-ec1c-415e-8765-a8d5c16ff33a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.139795 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f974-ec1c-415e-8765-a8d5c16ff33a None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.140438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14f974-ec1c-415e-8765-a8d5c16ff33a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:32.140769 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14f974-ec1c-415e-8765-a8d5c16ff33a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.142151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14f974-ec1c-415e-8765-a8d5c16ff33a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.158799 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 195/391] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:32.158799 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14f974-ec1c-415e-8765-a8d5c16ff33a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.175293 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:32.196372 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca65067-f3c7-4192-bc73-acf4ab28b322 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.228479 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:32.234503 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5633f7-9a94-4fe1-a3b1-0f75117b3abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.235145 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5633f7-9a94-4fe1-a3b1-0f75117b3abc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.235424 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5633f7-9a94-4fe1-a3b1-0f75117b3abc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.235696 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5633f7-9a94-4fe1-a3b1-0f75117b3abc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.243582 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5633f7-9a94-4fe1-a3b1-0f75117b3abc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.245065 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5633f7-9a94-4fe1-a3b1-0f75117b3abc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-5d5633f7-9a94-4fe1-a3b1-0f75117b3abc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:32.257837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:32.258673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:32.259498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:32.259498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:32.259498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:32.259498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:32.259498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:32.259498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:32.259498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:32.259498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:32.259498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 197/392] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:32.260004 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5633f7-9a94-4fe1-a3b1-0f75117b3abc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.271207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14f974-ec1c-415e-8765-a8d5c16ff33a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.307285 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935e747-f2c9-457d-b409-583a975aacae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.307914 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935e747-f2c9-457d-b409-583a975aacae None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.308184 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935e747-f2c9-457d-b409-583a975aacae None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.308461 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935e747-f2c9-457d-b409-583a975aacae None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.315535 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935e747-f2c9-457d-b409-583a975aacae None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.317002 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935e747-f2c9-457d-b409-583a975aacae None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8935e747-f2c9-457d-b409-583a975aacae None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:32.329744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:32.331158 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:32.332048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:32.332048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:32.332048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:32.332048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:32.332048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:32.332048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:32.332048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:32.332048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:32.332048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:14:32.332048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:32.332048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 196/393] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:32.332635 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935e747-f2c9-457d-b409-583a975aacae None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.368055 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5633f7-9a94-4fe1-a3b1-0f75117b3abc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.400854 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb94ec-2b0d-4676-96ff-629620ae5769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.401672 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb94ec-2b0d-4676-96ff-629620ae5769 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.401946 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb94ec-2b0d-4676-96ff-629620ae5769 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.402233 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb94ec-2b0d-4676-96ff-629620ae5769 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.409571 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb94ec-2b0d-4676-96ff-629620ae5769 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.411002 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb94ec-2b0d-4676-96ff-629620ae5769 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.413510 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-33fb94ec-2b0d-4676-96ff-629620ae5769 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:32.423119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:32.424721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 198/394] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:32.425615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb94ec-2b0d-4676-96ff-629620ae5769 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.437655 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935e747-f2c9-457d-b409-583a975aacae None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.463147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:32.472586 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c64ed-cc10-4781-8f10-03401b3075b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.473369 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c64ed-cc10-4781-8f10-03401b3075b3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.474017 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c64ed-cc10-4781-8f10-03401b3075b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.474291 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c64ed-cc10-4781-8f10-03401b3075b3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.481654 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c64ed-cc10-4781-8f10-03401b3075b3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.483155 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c64ed-cc10-4781-8f10-03401b3075b3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7b9c64ed-cc10-4781-8f10-03401b3075b3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:32.494613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:32.495720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 197/395] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:32.497096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c64ed-cc10-4781-8f10-03401b3075b3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.530340 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb94ec-2b0d-4676-96ff-629620ae5769 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.562747 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b2ba9-3380-4973-ac7a-ffbba0fea0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.563343 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b2ba9-3380-4973-ac7a-ffbba0fea0fe None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.563617 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b2ba9-3380-4973-ac7a-ffbba0fea0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.563885 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b2ba9-3380-4973-ac7a-ffbba0fea0fe None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.564589 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b2ba9-3380-4973-ac7a-ffbba0fea0fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:32.565479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b2ba9-3380-4973-ac7a-ffbba0fea0fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.566895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b2ba9-3380-4973-ac7a-ffbba0fea0fe None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.584639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 199/396] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:32.585770 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b2ba9-3380-4973-ac7a-ffbba0fea0fe None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.598756 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c64ed-cc10-4781-8f10-03401b3075b3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.631462 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f417619-907d-48ce-a2a8-10b3220cc3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.632061 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f417619-907d-48ce-a2a8-10b3220cc3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.632341 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f417619-907d-48ce-a2a8-10b3220cc3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.632648 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f417619-907d-48ce-a2a8-10b3220cc3d2 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.633374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f417619-907d-48ce-a2a8-10b3220cc3d2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:32.633690 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f417619-907d-48ce-a2a8-10b3220cc3d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.635141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f417619-907d-48ce-a2a8-10b3220cc3d2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.647258 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:32.653434 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 198/397] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:32.654085 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f417619-907d-48ce-a2a8-10b3220cc3d2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.692525 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b2ba9-3380-4973-ac7a-ffbba0fea0fe None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.698082 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:32.726352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced9dfe-9880-4a6e-9af8-060dcc781200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.726984 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced9dfe-9880-4a6e-9af8-060dcc781200 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.727272 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced9dfe-9880-4a6e-9af8-060dcc781200 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.727544 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced9dfe-9880-4a6e-9af8-060dcc781200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.737374 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced9dfe-9880-4a6e-9af8-060dcc781200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.738816 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced9dfe-9880-4a6e-9af8-060dcc781200 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.750852 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f417619-907d-48ce-a2a8-10b3220cc3d2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9ced9dfe-9880-4a6e-9af8-060dcc781200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:32.753499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:32.754431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:32.756420 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:32.756420 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:32.756420 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:32.756420 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 200/398] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:32.756420 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced9dfe-9880-4a6e-9af8-060dcc781200 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.786638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711cffe-b0fe-4b16-8185-87ce7bf91022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.787253 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711cffe-b0fe-4b16-8185-87ce7bf91022 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.787577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711cffe-b0fe-4b16-8185-87ce7bf91022 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.787812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711cffe-b0fe-4b16-8185-87ce7bf91022 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.788558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711cffe-b0fe-4b16-8185-87ce7bf91022 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:32.788871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711cffe-b0fe-4b16-8185-87ce7bf91022 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.790306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711cffe-b0fe-4b16-8185-87ce7bf91022 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.806759 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 199/399] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:32.807526 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711cffe-b0fe-4b16-8185-87ce7bf91022 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.861873 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced9dfe-9880-4a6e-9af8-060dcc781200 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.873982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:32.895146 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aef248-b9a2-4348-add8-d5fcff25d6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.895684 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aef248-b9a2-4348-add8-d5fcff25d6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.895968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aef248-b9a2-4348-add8-d5fcff25d6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.896241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aef248-b9a2-4348-add8-d5fcff25d6f9 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.896990 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aef248-b9a2-4348-add8-d5fcff25d6f9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:32.897296 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aef248-b9a2-4348-add8-d5fcff25d6f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.898757 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aef248-b9a2-4348-add8-d5fcff25d6f9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.904785 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711cffe-b0fe-4b16-8185-87ce7bf91022 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:32.906274 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:32.939322 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 201/400] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:32.939322 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aef248-b9a2-4348-add8-d5fcff25d6f9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:32.939586 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b498ed0-00a7-44c4-b379-6df907b12f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:32.939586 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b498ed0-00a7-44c4-b379-6df907b12f55 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:32.939586 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b498ed0-00a7-44c4-b379-6df907b12f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:32.940382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b498ed0-00a7-44c4-b379-6df907b12f55 None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/00ab674b018e474ab363e8e17d225e27/roles/d276003aacd44f11a8b5e733a9a54f02` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:32.955193 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b498ed0-00a7-44c4-b379-6df907b12f55 None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=00ab674b018e474ab363e8e17d225e27, role_id=d276003aacd44f11a8b5e733a9a54f02)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:32.957262 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b498ed0-00a7-44c4-b379-6df907b12f55 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:32.969513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 200/401] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:32 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/00ab674b018e474ab363e8e17d225e27/roles/d276003aacd44f11a8b5e733a9a54f02 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:32.970455 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b498ed0-00a7-44c4-b379-6df907b12f55 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.051503 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aef248-b9a2-4348-add8-d5fcff25d6f9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.092626 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78082b79-6562-439c-ba69-1b502311fa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.093175 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b498ed0-00a7-44c4-b379-6df907b12f55 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.093296 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78082b79-6562-439c-ba69-1b502311fa24 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.093530 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78082b79-6562-439c-ba69-1b502311fa24 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.093802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78082b79-6562-439c-ba69-1b502311fa24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.106113 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78082b79-6562-439c-ba69-1b502311fa24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.107646 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78082b79-6562-439c-ba69-1b502311fa24 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-78082b79-6562-439c-ba69-1b502311fa24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:33.124963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:33.126119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:33.126950 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:33.126950 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:33.126950 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:33.126950 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 202/402] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:33.126950 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78082b79-6562-439c-ba69-1b502311fa24 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.138300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b73d4-3e0d-426c-8bad-f1b9f94cad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.138899 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b73d4-3e0d-426c-8bad-f1b9f94cad3a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.139182 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b73d4-3e0d-426c-8bad-f1b9f94cad3a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.139452 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b73d4-3e0d-426c-8bad-f1b9f94cad3a None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.140102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b73d4-3e0d-426c-8bad-f1b9f94cad3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:33.140402 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b73d4-3e0d-426c-8bad-f1b9f94cad3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.141782 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b73d4-3e0d-426c-8bad-f1b9f94cad3a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.158348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 201/403] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:33.158348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b73d4-3e0d-426c-8bad-f1b9f94cad3a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.159643 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.173365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.242843 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78082b79-6562-439c-ba69-1b502311fa24 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.269491 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b73d4-3e0d-426c-8bad-f1b9f94cad3a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.278679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab623f69-27a2-4624-9ccb-5b9666509c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.279615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab623f69-27a2-4624-9ccb-5b9666509c75 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.279895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab623f69-27a2-4624-9ccb-5b9666509c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.280166 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab623f69-27a2-4624-9ccb-5b9666509c75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab623f69-27a2-4624-9ccb-5b9666509c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.293657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab623f69-27a2-4624-9ccb-5b9666509c75 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.303773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a403e-9638-46a2-9fd7-d05fef82be9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.304405 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a403e-9638-46a2-9fd7-d05fef82be9f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.304713 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a403e-9638-46a2-9fd7-d05fef82be9f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.304986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a403e-9638-46a2-9fd7-d05fef82be9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ab623f69-27a2-4624-9ccb-5b9666509c75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:33.309597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:33.310587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:33.311416 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:33.311416 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:33.311416 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:33.315274 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 203/404] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:33.316079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab623f69-27a2-4624-9ccb-5b9666509c75 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.316208 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a403e-9638-46a2-9fd7-d05fef82be9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.317543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a403e-9638-46a2-9fd7-d05fef82be9f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3a7a403e-9638-46a2-9fd7-d05fef82be9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:33.333254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:33.334166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:33.335018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:33.335018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:33.335018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:33.335018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 202/405] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:33.335239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a403e-9638-46a2-9fd7-d05fef82be9f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.397572 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.402605 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.429390 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab623f69-27a2-4624-9ccb-5b9666509c75 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.439186 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a403e-9638-46a2-9fd7-d05fef82be9f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.470125 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8fad2-2f22-40a9-bbe7-938279bc7ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.470756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8fad2-2f22-40a9-bbe7-938279bc7ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.471579 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8fad2-2f22-40a9-bbe7-938279bc7ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.471901 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8fad2-2f22-40a9-bbe7-938279bc7ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.472484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8fad2-2f22-40a9-bbe7-938279bc7ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:33.472826 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8fad2-2f22-40a9-bbe7-938279bc7ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.474427 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8fad2-2f22-40a9-bbe7-938279bc7ed4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.479701 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fab46-3fe2-409d-9ed2-e35b715a3cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.480311 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fab46-3fe2-409d-9ed2-e35b715a3cd7 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.480628 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fab46-3fe2-409d-9ed2-e35b715a3cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.480904 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fab46-3fe2-409d-9ed2-e35b715a3cd7 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.481548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fab46-3fe2-409d-9ed2-e35b715a3cd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.483434 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fab46-3fe2-409d-9ed2-e35b715a3cd7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.491659 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 204/406] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:33.491659 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8fad2-2f22-40a9-bbe7-938279bc7ed4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.508336 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 203/407] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:33 2022] POST /identity/v3/users => generated 247 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:33.509013 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fab46-3fe2-409d-9ed2-e35b715a3cd7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.635833 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8fad2-2f22-40a9-bbe7-938279bc7ed4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.641817 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fab46-3fe2-409d-9ed2-e35b715a3cd7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.671437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.690904 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3b59f-3912-48ca-a677-b0fb867bebf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.691086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c451eb-7c8b-451f-8066-f6e5ef1b50c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.691680 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c451eb-7c8b-451f-8066-f6e5ef1b50c0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.691956 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c451eb-7c8b-451f-8066-f6e5ef1b50c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.692223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c451eb-7c8b-451f-8066-f6e5ef1b50c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.692633 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b59f-3912-48ca-a677-b0fb867bebf6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.692920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c451eb-7c8b-451f-8066-f6e5ef1b50c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:33.693218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c451eb-7c8b-451f-8066-f6e5ef1b50c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.693475 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b59f-3912-48ca-a677-b0fb867bebf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.693757 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b59f-3912-48ca-a677-b0fb867bebf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.694408 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3b59f-3912-48ca-a677-b0fb867bebf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:33.694604 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c451eb-7c8b-451f-8066-f6e5ef1b50c0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.694699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3b59f-3912-48ca-a677-b0fb867bebf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.696288 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3b59f-3912-48ca-a677-b0fb867bebf6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.698429 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.709412 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 204/408] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:33.710374 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 205/409] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:33.710792 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c451eb-7c8b-451f-8066-f6e5ef1b50c0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.715588 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3b59f-3912-48ca-a677-b0fb867bebf6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.832964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c451eb-7c8b-451f-8066-f6e5ef1b50c0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.847857 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3b59f-3912-48ca-a677-b0fb867bebf6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:33.871128 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f907acd-c25d-4b68-be2d-2c7bce0cefdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.871980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f907acd-c25d-4b68-be2d-2c7bce0cefdb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.872344 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f907acd-c25d-4b68-be2d-2c7bce0cefdb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.872707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f907acd-c25d-4b68-be2d-2c7bce0cefdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.880661 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f907acd-c25d-4b68-be2d-2c7bce0cefdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.882240 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f907acd-c25d-4b68-be2d-2c7bce0cefdb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.891296 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c0c52-0e00-4cc7-8d25-2b9fda2a307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:33.891934 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c0c52-0e00-4cc7-8d25-2b9fda2a307b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.892215 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c0c52-0e00-4cc7-8d25-2b9fda2a307b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.892487 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c0c52-0e00-4cc7-8d25-2b9fda2a307b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9f907acd-c25d-4b68-be2d-2c7bce0cefdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:33.895290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:33.896086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:33.896963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:33.896963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:33.896963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:33.896963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:33.896963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:33.896963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:33.896963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:33.896963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:33.896963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 205/410] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:33.897809 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f907acd-c25d-4b68-be2d-2c7bce0cefdb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.900274 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c0c52-0e00-4cc7-8d25-2b9fda2a307b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:33.901918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c0c52-0e00-4cc7-8d25-2b9fda2a307b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-402c0c52-0e00-4cc7-8d25-2b9fda2a307b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:33.916942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:33.918598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:33.920086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:33.920086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:33.920086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:33.920086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:33.920086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:33.920086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:33.920086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:33.920086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:33.920086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 206/411] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:33.920086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c0c52-0e00-4cc7-8d25-2b9fda2a307b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:33.952654 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:33.981849 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:34.024715 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f907acd-c25d-4b68-be2d-2c7bce0cefdb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.040147 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c0c52-0e00-4cc7-8d25-2b9fda2a307b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.061195 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e07df-a802-4c64-a17f-8cab3a43b608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.062204 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e07df-a802-4c64-a17f-8cab3a43b608 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.062659 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e07df-a802-4c64-a17f-8cab3a43b608 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.063134 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e07df-a802-4c64-a17f-8cab3a43b608 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.070749 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e07df-a802-4c64-a17f-8cab3a43b608 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.072391 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e07df-a802-4c64-a17f-8cab3a43b608 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.081577 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78676b-4aa7-4878-ab09-68c7be5f86d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.082269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78676b-4aa7-4878-ab09-68c7be5f86d3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.082621 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78676b-4aa7-4878-ab09-68c7be5f86d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.082965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78676b-4aa7-4878-ab09-68c7be5f86d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.083721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78676b-4aa7-4878-ab09-68c7be5f86d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:34.084097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78676b-4aa7-4878-ab09-68c7be5f86d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.085631 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78676b-4aa7-4878-ab09-68c7be5f86d3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-930e07df-a802-4c64-a17f-8cab3a43b608 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:34.086030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:34.087212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:34.088052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:34.088052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:34.088052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:34.088052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:34.088052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:34.088052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:34.088052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:34.088052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:34.090041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 206/412] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:34.091033 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e07df-a802-4c64-a17f-8cab3a43b608 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:34.099794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 207/413] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:34.101213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78676b-4aa7-4878-ab09-68c7be5f86d3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:34.215665 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e07df-a802-4c64-a17f-8cab3a43b608 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.221828 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78676b-4aa7-4878-ab09-68c7be5f86d3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.224870 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:34.240869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:34.259204 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4a7ec-f1ac-4adc-9be5-3e81fd122992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.260096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4a7ec-f1ac-4adc-9be5-3e81fd122992 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.260554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4a7ec-f1ac-4adc-9be5-3e81fd122992 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.261294 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4a7ec-f1ac-4adc-9be5-3e81fd122992 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.262910 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4a7ec-f1ac-4adc-9be5-3e81fd122992 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:34.263366 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4a7ec-f1ac-4adc-9be5-3e81fd122992 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.265261 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4a7ec-f1ac-4adc-9be5-3e81fd122992 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.266983 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ca9ff-980f-4fa4-8d94-da1e84a3fc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.268281 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ca9ff-980f-4fa4-8d94-da1e84a3fc64 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.268431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ca9ff-980f-4fa4-8d94-da1e84a3fc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.268843 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ca9ff-980f-4fa4-8d94-da1e84a3fc64 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.269641 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ca9ff-980f-4fa4-8d94-da1e84a3fc64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:34.270034 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ca9ff-980f-4fa4-8d94-da1e84a3fc64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.272081 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ca9ff-980f-4fa4-8d94-da1e84a3fc64 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.286646 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 207/415] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:34.286834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 208/415] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:34.287763 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ca9ff-980f-4fa4-8d94-da1e84a3fc64 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:34.293207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4a7ec-f1ac-4adc-9be5-3e81fd122992 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:34.417506 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4a7ec-f1ac-4adc-9be5-3e81fd122992 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.419125 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ca9ff-980f-4fa4-8d94-da1e84a3fc64 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.460884 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907ffa8-acdb-4b60-b07e-50c40d7c2a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.462348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907ffa8-acdb-4b60-b07e-50c40d7c2a8b None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.462890 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907ffa8-acdb-4b60-b07e-50c40d7c2a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.463458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907ffa8-acdb-4b60-b07e-50c40d7c2a8b None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/7a5b837bc9c04fc5a53077d152f4d6d5/roles/d276003aacd44f11a8b5e733a9a54f02` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.465593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291718f-7b6f-42ca-be06-3731cab66a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.466409 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291718f-7b6f-42ca-be06-3731cab66a7e None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.466690 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291718f-7b6f-42ca-be06-3731cab66a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.467051 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291718f-7b6f-42ca-be06-3731cab66a7e None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/e0f9d9dc1f7143e0be709e17d1a5f484/roles/d276003aacd44f11a8b5e733a9a54f02` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.481831 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907ffa8-acdb-4b60-b07e-50c40d7c2a8b None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=7a5b837bc9c04fc5a53077d152f4d6d5, role_id=d276003aacd44f11a8b5e733a9a54f02)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.484391 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907ffa8-acdb-4b60-b07e-50c40d7c2a8b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.486320 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291718f-7b6f-42ca-be06-3731cab66a7e None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=e0f9d9dc1f7143e0be709e17d1a5f484, role_id=d276003aacd44f11a8b5e733a9a54f02)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.488707 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291718f-7b6f-42ca-be06-3731cab66a7e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.501317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 208/416] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:34 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/7a5b837bc9c04fc5a53077d152f4d6d5/roles/d276003aacd44f11a8b5e733a9a54f02 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:34.502355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907ffa8-acdb-4b60-b07e-50c40d7c2a8b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:34.504357 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 209/417] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:34 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/e0f9d9dc1f7143e0be709e17d1a5f484/roles/d276003aacd44f11a8b5e733a9a54f02 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:34.517509 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:34.624353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907ffa8-acdb-4b60-b07e-50c40d7c2a8b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.667754 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2aa658-e6b4-4848-ab83-1299f638ac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.668992 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2aa658-e6b4-4848-ab83-1299f638ac3b None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.669485 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2aa658-e6b4-4848-ab83-1299f638ac3b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.669966 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2aa658-e6b4-4848-ab83-1299f638ac3b None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/36450acd551f40d08a4bc21c9a9c8877/roles/d276003aacd44f11a8b5e733a9a54f02` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.688102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2aa658-e6b4-4848-ab83-1299f638ac3b None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=36450acd551f40d08a4bc21c9a9c8877, role_id=d276003aacd44f11a8b5e733a9a54f02)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.690131 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2aa658-e6b4-4848-ab83-1299f638ac3b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.707859 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 209/418] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:34 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/36450acd551f40d08a4bc21c9a9c8877/roles/d276003aacd44f11a8b5e733a9a54f02 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:35.381850 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169b8c9-5424-4e16-8238-8273769af523 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:35.382228 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169b8c9-5424-4e16-8238-8273769af523 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:35.382404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169b8c9-5424-4e16-8238-8273769af523 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:35.384018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 210/419] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:35.389626 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9efc5-01f6-44e2-b4f4-8678aa7a1f89 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:35.389917 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9efc5-01f6-44e2-b4f4-8678aa7a1f89 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:35.390198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9efc5-01f6-44e2-b4f4-8678aa7a1f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:35.417505 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-42a9efc5-01f6-44e2-b4f4-8678aa7a1f89 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:35.418714 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-42a9efc5-01f6-44e2-b4f4-8678aa7a1f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:35.499154 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:35.515232 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 210/420] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:35.885945 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169b8c9-5424-4e16-8238-8273769af523 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:35.913342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04549c3-2a4e-43a1-aef7-b25137a10af9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:35.913558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04549c3-2a4e-43a1-aef7-b25137a10af9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:35.913689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:35.913839 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04549c3-2a4e-43a1-aef7-b25137a10af9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:35.915337 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 211/421] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:35.930261 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3fe71-8088-4f1d-afce-79bee0e69c55 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:35.930539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3fe71-8088-4f1d-afce-79bee0e69c55 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:35.930813 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3fe71-8088-4f1d-afce-79bee0e69c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:35.965518 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-80a3fe71-8088-4f1d-afce-79bee0e69c55 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:35.966711 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-80a3fe71-8088-4f1d-afce-79bee0e69c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:35.997578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169b8c9-5424-4e16-8238-8273769af523 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:36.063520 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25090e9-0292-4ae7-8ef3-b319a0965832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:36.064471 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25090e9-0292-4ae7-8ef3-b319a0965832 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.065370 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25090e9-0292-4ae7-8ef3-b319a0965832 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.067063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25090e9-0292-4ae7-8ef3-b319a0965832 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.087089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25090e9-0292-4ae7-8ef3-b319a0965832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a25090e9-0292-4ae7-8ef3-b319a0965832 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:36.094731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:36.095586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:36.096514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:36.100994 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 211/422] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:36.103327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 212/423] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:36.104879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25090e9-0292-4ae7-8ef3-b319a0965832 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:36.222535 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:36.242745 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25090e9-0292-4ae7-8ef3-b319a0965832 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:36.282232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6ccfd-67c6-4cb7-86a5-ab50905e1193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:36.282889 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6ccfd-67c6-4cb7-86a5-ab50905e1193 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.283171 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6ccfd-67c6-4cb7-86a5-ab50905e1193 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.283464 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6ccfd-67c6-4cb7-86a5-ab50905e1193 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.284132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6ccfd-67c6-4cb7-86a5-ab50905e1193 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:36.284428 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6ccfd-67c6-4cb7-86a5-ab50905e1193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:36.285812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6ccfd-67c6-4cb7-86a5-ab50905e1193 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:36.295078 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 212/424] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:36.301099 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3fe71-8088-4f1d-afce-79bee0e69c55 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:36.414416 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6ccfd-67c6-4cb7-86a5-ab50905e1193 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:36.430220 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3fe71-8088-4f1d-afce-79bee0e69c55 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:36.468895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:36.498255 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01fd2c-a89a-4fd9-bac5-c7b5155876b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:36.498815 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01fd2c-a89a-4fd9-bac5-c7b5155876b1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.499093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01fd2c-a89a-4fd9-bac5-c7b5155876b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.499361 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01fd2c-a89a-4fd9-bac5-c7b5155876b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.526406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01fd2c-a89a-4fd9-bac5-c7b5155876b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:36.527870 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01fd2c-a89a-4fd9-bac5-c7b5155876b1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:36.556970 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6ccfd-67c6-4cb7-86a5-ab50905e1193 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5e01fd2c-a89a-4fd9-bac5-c7b5155876b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:36.566111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:36.567000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:36.567845 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:36.567845 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:36.567845 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:36.567845 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 213/425] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:36.573056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01fd2c-a89a-4fd9-bac5-c7b5155876b1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:36.593058 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7596c-0e16-4087-a253-0778b34a8c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:36.593691 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7596c-0e16-4087-a253-0778b34a8c77 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.593972 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7596c-0e16-4087-a253-0778b34a8c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.594247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7596c-0e16-4087-a253-0778b34a8c77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.605965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7596c-0e16-4087-a253-0778b34a8c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-22d7596c-0e16-4087-a253-0778b34a8c77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:36.614336 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:36.615397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:36.616392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:36.619024 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 213/426] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:36.630166 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7596c-0e16-4087-a253-0778b34a8c77 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:36.640377 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:36.797379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01fd2c-a89a-4fd9-bac5-c7b5155876b1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:36.828185 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7596c-0e16-4087-a253-0778b34a8c77 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:36.869760 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f5119-2e7d-4673-89c1-23c150f3214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:36.870671 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f5119-2e7d-4673-89c1-23c150f3214f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.871285 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f5119-2e7d-4673-89c1-23c150f3214f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.872737 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f5119-2e7d-4673-89c1-23c150f3214f None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.873582 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f5119-2e7d-4673-89c1-23c150f3214f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:36.874227 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f5119-2e7d-4673-89c1-23c150f3214f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:36.876873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f5119-2e7d-4673-89c1-23c150f3214f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:36.899446 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 214/427] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:36.905957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c20e34-1706-47e1-baef-164b88a42c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:36.906581 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c20e34-1706-47e1-baef-164b88a42c9d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.906800 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f5119-2e7d-4673-89c1-23c150f3214f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:36.906901 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c20e34-1706-47e1-baef-164b88a42c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.907125 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c20e34-1706-47e1-baef-164b88a42c9d None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.908606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c20e34-1706-47e1-baef-164b88a42c9d None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:36.908606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c20e34-1706-47e1-baef-164b88a42c9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:36.910051 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c20e34-1706-47e1-baef-164b88a42c9d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:36.927926 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 214/428] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:36.932902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c20e34-1706-47e1-baef-164b88a42c9d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:36.944390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.060508 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f5119-2e7d-4673-89c1-23c150f3214f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.063287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.116202 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c20e34-1706-47e1-baef-164b88a42c9d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.125746 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6737201-f795-42bb-8531-31e31e26dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.127104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6737201-f795-42bb-8531-31e31e26dd45 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.128130 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6737201-f795-42bb-8531-31e31e26dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.128641 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6737201-f795-42bb-8531-31e31e26dd45 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.138428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6737201-f795-42bb-8531-31e31e26dd45 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.140381 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6737201-f795-42bb-8531-31e31e26dd45 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b6737201-f795-42bb-8531-31e31e26dd45 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:37.156495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:37.157450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:37.158352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:37.158352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:37.158352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:37.158352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:37.158352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:37.158352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:37.158352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:37.158352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:37.158352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:14:37.158352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:37.159385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eccde-c23a-4657-bbb2-f73064b04699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.159385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eccde-c23a-4657-bbb2-f73064b04699 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.159385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eccde-c23a-4657-bbb2-f73064b04699 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.159385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eccde-c23a-4657-bbb2-f73064b04699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.162089 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 215/429] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Nov 2 07:14:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:37.163449 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43d7f7-fa54-4951-a007-6832e6aa201f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.163889 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43d7f7-fa54-4951-a007-6832e6aa201f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.164336 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43d7f7-fa54-4951-a007-6832e6aa201f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.165990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 216/430] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:37.166964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43d7f7-fa54-4951-a007-6832e6aa201f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.181679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eccde-c23a-4657-bbb2-f73064b04699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.183864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eccde-c23a-4657-bbb2-f73064b04699 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-501eccde-c23a-4657-bbb2-f73064b04699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:37.203748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:37.204733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:37.205663 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:37.205663 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:37.205663 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:37.208688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 215/431] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:37.210078 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf18ba-646d-4269-bdbe-5c82e7b237cf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.210361 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf18ba-646d-4269-bdbe-5c82e7b237cf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.210635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf18ba-646d-4269-bdbe-5c82e7b237cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.231477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-86bf18ba-646d-4269-bdbe-5c82e7b237cf None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:37.232769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-86bf18ba-646d-4269-bdbe-5c82e7b237cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:37.295025 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.308923 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43d7f7-fa54-4951-a007-6832e6aa201f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.309671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 216/432] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:37.310577 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf18ba-646d-4269-bdbe-5c82e7b237cf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.349584 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa80dd-c73a-4356-a018-4347a26b428a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.350572 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa80dd-c73a-4356-a018-4347a26b428a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.351087 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa80dd-c73a-4356-a018-4347a26b428a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.351657 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa80dd-c73a-4356-a018-4347a26b428a None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.352648 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa80dd-c73a-4356-a018-4347a26b428a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:37.353231 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa80dd-c73a-4356-a018-4347a26b428a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.354986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa80dd-c73a-4356-a018-4347a26b428a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.362332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.378504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 217/433] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:37.384356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa80dd-c73a-4356-a018-4347a26b428a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.414556 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf18ba-646d-4269-bdbe-5c82e7b237cf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.450969 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79dd08b-db43-448f-85cb-bd66ca295fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.451576 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dd08b-db43-448f-85cb-bd66ca295fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.451857 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dd08b-db43-448f-85cb-bd66ca295fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.452131 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dd08b-db43-448f-85cb-bd66ca295fc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.452802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79dd08b-db43-448f-85cb-bd66ca295fc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:37.453101 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79dd08b-db43-448f-85cb-bd66ca295fc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.454481 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79dd08b-db43-448f-85cb-bd66ca295fc6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.466086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 217/434] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:37.467448 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349908d-e1b9-4560-b086-957e9330ef9a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.467736 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349908d-e1b9-4560-b086-957e9330ef9a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.468010 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349908d-e1b9-4560-b086-957e9330ef9a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.469495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 218/435] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:37.470559 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9925c6-5bfa-4c2c-a7c3-c2f18e6ea79f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.470805 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9925c6-5bfa-4c2c-a7c3-c2f18e6ea79f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.471611 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9925c6-5bfa-4c2c-a7c3-c2f18e6ea79f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.480348 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 219/436] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:37.480348 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9925c6-5bfa-4c2c-a7c3-c2f18e6ea79f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.504209 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa80dd-c73a-4356-a018-4347a26b428a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.548146 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6a528-e9d4-4c2e-bf40-7807d01bd74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.548871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6a528-e9d4-4c2e-bf40-7807d01bd74b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.549676 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6a528-e9d4-4c2e-bf40-7807d01bd74b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.549978 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6a528-e9d4-4c2e-bf40-7807d01bd74b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.570826 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.582981 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9925c6-5bfa-4c2c-a7c3-c2f18e6ea79f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.593673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.618094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581a272-7134-408f-aae2-a07d0f2ae502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.618726 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581a272-7134-408f-aae2-a07d0f2ae502 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.619002 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581a272-7134-408f-aae2-a07d0f2ae502 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.619269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581a272-7134-408f-aae2-a07d0f2ae502 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.626832 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581a272-7134-408f-aae2-a07d0f2ae502 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.628041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6a528-e9d4-4c2e-bf40-7807d01bd74b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.628307 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581a272-7134-408f-aae2-a07d0f2ae502 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.629526 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6a528-e9d4-4c2e-bf40-7807d01bd74b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b581a272-7134-408f-aae2-a07d0f2ae502 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:37.639200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:37.640117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:37.641152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:37.641152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:37.641152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:37.641152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:37.641152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:37.641152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:37.641152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:37.641152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:37.641152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:14:37.641152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:37.641152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 220/437] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:37.641860 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f0c4f-16c5-4854-9a99-65d91f13bf02 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.642146 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f0c4f-16c5-4854-9a99-65d91f13bf02 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.642416 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f0c4f-16c5-4854-9a99-65d91f13bf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.661222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7f7f0c4f-16c5-4854-9a99-65d91f13bf02 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:37.662390 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7f7f0c4f-16c5-4854-9a99-65d91f13bf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:37.728036 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 221/438] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:37.729287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca79c4a-b527-43a4-b8ca-33fcf6909a34 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.729576 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca79c4a-b527-43a4-b8ca-33fcf6909a34 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.729851 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca79c4a-b527-43a4-b8ca-33fcf6909a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.741215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 218/439] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:14:37.742333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6a528-e9d4-4c2e-bf40-7807d01bd74b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.752431 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7ca79c4a-b527-43a4-b8ca-33fcf6909a34 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:37.753688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7ca79c4a-b527-43a4-b8ca-33fcf6909a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:37.795676 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.820124 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 222/440] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:37.820881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca79c4a-b527-43a4-b8ca-33fcf6909a34 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.851777 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.858035 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6a528-e9d4-4c2e-bf40-7807d01bd74b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.894521 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e91022-3149-457d-afc0-513cd55918e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.894521 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e91022-3149-457d-afc0-513cd55918e2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.894796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e91022-3149-457d-afc0-513cd55918e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.895079 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e91022-3149-457d-afc0-513cd55918e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.905519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e91022-3149-457d-afc0-513cd55918e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-04e91022-3149-457d-afc0-513cd55918e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:37.910320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:37.911206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 219/441] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:37.917044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e91022-3149-457d-afc0-513cd55918e2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.932964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca79c4a-b527-43a4-b8ca-33fcf6909a34 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.969079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c5095-8b39-4cf7-8999-976249d95cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.970063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c5095-8b39-4cf7-8999-976249d95cbd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.970535 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c5095-8b39-4cf7-8999-976249d95cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.971545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c5095-8b39-4cf7-8999-976249d95cbd None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.972458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c5095-8b39-4cf7-8999-976249d95cbd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:37.973009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c5095-8b39-4cf7-8999-976249d95cbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.974795 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c5095-8b39-4cf7-8999-976249d95cbd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.993339 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 223/442] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:37.994448 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c5095-8b39-4cf7-8999-976249d95cbd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.014568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e91022-3149-457d-afc0-513cd55918e2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.025945 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:38.053276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac166f-76eb-4bd6-ba72-3f0cb38088c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.054063 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac166f-76eb-4bd6-ba72-3f0cb38088c0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.054335 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac166f-76eb-4bd6-ba72-3f0cb38088c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.054614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac166f-76eb-4bd6-ba72-3f0cb38088c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.066010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac166f-76eb-4bd6-ba72-3f0cb38088c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:38.067498 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac166f-76eb-4bd6-ba72-3f0cb38088c0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d6ac166f-76eb-4bd6-ba72-3f0cb38088c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:38.083068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:38.083941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:38.084820 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:38.084820 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:38.084820 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:38.084820 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 220/443] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:38.085045 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac166f-76eb-4bd6-ba72-3f0cb38088c0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.087562 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:38.102334 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c5095-8b39-4cf7-8999-976249d95cbd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.136133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5f67b-8f10-48dd-aa78-72cae253fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.136762 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5f67b-8f10-48dd-aa78-72cae253fe27 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.137041 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5f67b-8f10-48dd-aa78-72cae253fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.137310 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5f67b-8f10-48dd-aa78-72cae253fe27 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.137960 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5f67b-8f10-48dd-aa78-72cae253fe27 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:38.138260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5f67b-8f10-48dd-aa78-72cae253fe27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:38.139610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5f67b-8f10-48dd-aa78-72cae253fe27 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:38.147968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 224/444] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:38.148790 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5f67b-8f10-48dd-aa78-72cae253fe27 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.194039 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac166f-76eb-4bd6-ba72-3f0cb38088c0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.230871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff80abb-ef8e-42cf-91f9-79bddb73a252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.231489 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff80abb-ef8e-42cf-91f9-79bddb73a252 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.231764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff80abb-ef8e-42cf-91f9-79bddb73a252 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.232032 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff80abb-ef8e-42cf-91f9-79bddb73a252 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.242704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff80abb-ef8e-42cf-91f9-79bddb73a252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:38.245699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5f67b-8f10-48dd-aa78-72cae253fe27 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7ff80abb-ef8e-42cf-91f9-79bddb73a252 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:38.247885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:38.248982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:38.249825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 221/445] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:38.250472 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff80abb-ef8e-42cf-91f9-79bddb73a252 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.277261 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:38.285304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6000ce6-f807-4974-b758-eee951199035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.285938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6000ce6-f807-4974-b758-eee951199035 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.286216 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6000ce6-f807-4974-b758-eee951199035 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.286492 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6000ce6-f807-4974-b758-eee951199035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.297301 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6000ce6-f807-4974-b758-eee951199035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:38.298754 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6000ce6-f807-4974-b758-eee951199035 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e6000ce6-f807-4974-b758-eee951199035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:38.313627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:38.314655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:38.315516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:38.315516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:38.315516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:38.315516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 225/446] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:38.315516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6000ce6-f807-4974-b758-eee951199035 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.318071 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:38.359723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff80abb-ef8e-42cf-91f9-79bddb73a252 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.393671 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9c108-cbd9-4662-ac16-42a750312071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.394272 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c108-cbd9-4662-ac16-42a750312071 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.394554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c108-cbd9-4662-ac16-42a750312071 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.394829 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c108-cbd9-4662-ac16-42a750312071 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.395477 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9c108-cbd9-4662-ac16-42a750312071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:38.395780 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9c108-cbd9-4662-ac16-42a750312071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:38.397182 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9c108-cbd9-4662-ac16-42a750312071 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:38.408961 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 222/447] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:38.409754 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9c108-cbd9-4662-ac16-42a750312071 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.413634 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6000ce6-f807-4974-b758-eee951199035 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.447023 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90bcd7-3f52-49a4-b404-2a1aadcc0aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.447644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90bcd7-3f52-49a4-b404-2a1aadcc0aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.447921 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90bcd7-3f52-49a4-b404-2a1aadcc0aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.448190 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90bcd7-3f52-49a4-b404-2a1aadcc0aa0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.458620 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90bcd7-3f52-49a4-b404-2a1aadcc0aa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8b90bcd7-3f52-49a4-b404-2a1aadcc0aa0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:38.464741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:38.465602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:38.466572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 226/448] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:38.467657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90bcd7-3f52-49a4-b404-2a1aadcc0aa0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.495380 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:38.513761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9c108-cbd9-4662-ac16-42a750312071 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.529265 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:38.547910 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82f286-edf8-46f2-b123-e82e3cef8f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.548544 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82f286-edf8-46f2-b123-e82e3cef8f27 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.548848 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82f286-edf8-46f2-b123-e82e3cef8f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.549648 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82f286-edf8-46f2-b123-e82e3cef8f27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.560315 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82f286-edf8-46f2-b123-e82e3cef8f27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:38.561836 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82f286-edf8-46f2-b123-e82e3cef8f27 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:38.567991 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90bcd7-3f52-49a4-b404-2a1aadcc0aa0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ae82f286-edf8-46f2-b123-e82e3cef8f27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:38.576223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:38.577279 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:38.583307 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:38.583307 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:38.583307 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:38.583307 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 223/449] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:38.583307 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82f286-edf8-46f2-b123-e82e3cef8f27 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.601949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacddbd-59b8-4c5e-bf45-a54b75b8f663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.602558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacddbd-59b8-4c5e-bf45-a54b75b8f663 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.602839 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacddbd-59b8-4c5e-bf45-a54b75b8f663 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.603112 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacddbd-59b8-4c5e-bf45-a54b75b8f663 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.603771 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacddbd-59b8-4c5e-bf45-a54b75b8f663 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:38.604071 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacddbd-59b8-4c5e-bf45-a54b75b8f663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:38.605470 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacddbd-59b8-4c5e-bf45-a54b75b8f663 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:38.613234 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 227/450] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:38.613957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacddbd-59b8-4c5e-bf45-a54b75b8f663 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.680600 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82f286-edf8-46f2-b123-e82e3cef8f27 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.714149 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f529780-e8ff-4ee4-9bd5-e660e8095655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.714750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529780-e8ff-4ee4-9bd5-e660e8095655 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.715029 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529780-e8ff-4ee4-9bd5-e660e8095655 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.715300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529780-e8ff-4ee4-9bd5-e660e8095655 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.715939 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f529780-e8ff-4ee4-9bd5-e660e8095655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:38.716241 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f529780-e8ff-4ee4-9bd5-e660e8095655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:38.716777 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacddbd-59b8-4c5e-bf45-a54b75b8f663 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.717644 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f529780-e8ff-4ee4-9bd5-e660e8095655 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:38.719808 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:38.731242 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 224/451] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:38.732044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f529780-e8ff-4ee4-9bd5-e660e8095655 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.741040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:38.753195 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d72faa-0373-4db3-85b6-c62c80c122db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.753853 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d72faa-0373-4db3-85b6-c62c80c122db None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.754135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d72faa-0373-4db3-85b6-c62c80c122db None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.754405 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d72faa-0373-4db3-85b6-c62c80c122db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.760996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d72faa-0373-4db3-85b6-c62c80c122db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:38.762429 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d72faa-0373-4db3-85b6-c62c80c122db None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d1d72faa-0373-4db3-85b6-c62c80c122db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:38.773968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:38.774801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:38.775823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:38.775823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:38.775823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:38.775823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:38.775823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:38.775823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:38.775823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:38.775823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:38.775823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 228/452] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:38.775823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d72faa-0373-4db3-85b6-c62c80c122db None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.832459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f529780-e8ff-4ee4-9bd5-e660e8095655 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.865863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf25c17-a2e5-4c62-a7d0-14ab195b3774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.866396 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25c17-a2e5-4c62-a7d0-14ab195b3774 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.866675 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25c17-a2e5-4c62-a7d0-14ab195b3774 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.866948 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25c17-a2e5-4c62-a7d0-14ab195b3774 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.867604 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf25c17-a2e5-4c62-a7d0-14ab195b3774 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:38.867904 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf25c17-a2e5-4c62-a7d0-14ab195b3774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:38.869290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf25c17-a2e5-4c62-a7d0-14ab195b3774 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:38.872964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d72faa-0373-4db3-85b6-c62c80c122db None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:38.877210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 225/453] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:38.878009 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf25c17-a2e5-4c62-a7d0-14ab195b3774 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.908350 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342c975-64a4-4a86-ad32-e0893f8c548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:38.909489 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342c975-64a4-4a86-ad32-e0893f8c548f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:38.909769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342c975-64a4-4a86-ad32-e0893f8c548f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:38.910041 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342c975-64a4-4a86-ad32-e0893f8c548f None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:38.910676 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342c975-64a4-4a86-ad32-e0893f8c548f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:38.910976 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342c975-64a4-4a86-ad32-e0893f8c548f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:38.912334 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342c975-64a4-4a86-ad32-e0893f8c548f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:38.923714 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 229/454] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:38.925079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342c975-64a4-4a86-ad32-e0893f8c548f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:38.934767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:38.969084 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:38.979478 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf25c17-a2e5-4c62-a7d0-14ab195b3774 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.012935 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd31239-0fa9-4c61-90cd-f1c7ddc963b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.013561 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd31239-0fa9-4c61-90cd-f1c7ddc963b5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.013846 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd31239-0fa9-4c61-90cd-f1c7ddc963b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.014112 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd31239-0fa9-4c61-90cd-f1c7ddc963b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.023383 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342c975-64a4-4a86-ad32-e0893f8c548f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.024820 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd31239-0fa9-4c61-90cd-f1c7ddc963b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.026191 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd31239-0fa9-4c61-90cd-f1c7ddc963b5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bfd31239-0fa9-4c61-90cd-f1c7ddc963b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:39.040441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:39.041348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:39.042164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:39.042164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:39.042164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:39.042164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 226/455] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:39.042370 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd31239-0fa9-4c61-90cd-f1c7ddc963b5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.058787 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7286e27-3dd9-43a1-ba04-0839e6d8a5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.059413 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7286e27-3dd9-43a1-ba04-0839e6d8a5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.059688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7286e27-3dd9-43a1-ba04-0839e6d8a5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.059956 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7286e27-3dd9-43a1-ba04-0839e6d8a5a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.066529 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7286e27-3dd9-43a1-ba04-0839e6d8a5a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.067964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7286e27-3dd9-43a1-ba04-0839e6d8a5a1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e7286e27-3dd9-43a1-ba04-0839e6d8a5a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:39.078802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:39.079809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:39.081204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:39.081204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:39.081204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:39.081204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:39.081204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:39.081204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:39.081204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:39.081204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:39.081204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 230/456] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:39.081204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7286e27-3dd9-43a1-ba04-0839e6d8a5a1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.140678 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd31239-0fa9-4c61-90cd-f1c7ddc963b5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.149674 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:39.173770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f2c25-a962-4dab-b1c6-3f5f904aa9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.174881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f2c25-a962-4dab-b1c6-3f5f904aa9cc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.175155 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f2c25-a962-4dab-b1c6-3f5f904aa9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.175427 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f2c25-a962-4dab-b1c6-3f5f904aa9cc None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.176106 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f2c25-a962-4dab-b1c6-3f5f904aa9cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:39.176407 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f2c25-a962-4dab-b1c6-3f5f904aa9cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.177858 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f2c25-a962-4dab-b1c6-3f5f904aa9cc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.179038 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7286e27-3dd9-43a1-ba04-0839e6d8a5a1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.181397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:39.190305 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 227/457] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:39.191229 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f2c25-a962-4dab-b1c6-3f5f904aa9cc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.214181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc5baa-ab7d-427a-912e-343a2bd9fbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.214798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc5baa-ab7d-427a-912e-343a2bd9fbd6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.215077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc5baa-ab7d-427a-912e-343a2bd9fbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.215341 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc5baa-ab7d-427a-912e-343a2bd9fbd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.225517 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc5baa-ab7d-427a-912e-343a2bd9fbd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.226959 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc5baa-ab7d-427a-912e-343a2bd9fbd6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-08fc5baa-ab7d-427a-912e-343a2bd9fbd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:39.241347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:39.242299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:39.243532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:39.243532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:39.243532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:39.243532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 231/458] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:39.243532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc5baa-ab7d-427a-912e-343a2bd9fbd6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.297319 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f2c25-a962-4dab-b1c6-3f5f904aa9cc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.331992 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0783d0-8f74-414f-85da-c56cd5e4a298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.332649 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0783d0-8f74-414f-85da-c56cd5e4a298 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.332935 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0783d0-8f74-414f-85da-c56cd5e4a298 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.333213 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0783d0-8f74-414f-85da-c56cd5e4a298 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.340667 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0783d0-8f74-414f-85da-c56cd5e4a298 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.342091 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0783d0-8f74-414f-85da-c56cd5e4a298 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.342302 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc5baa-ab7d-427a-912e-343a2bd9fbd6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-dc0783d0-8f74-414f-85da-c56cd5e4a298 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:39.353300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:39.354365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:39.355266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:39.355266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:39.355266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:39.355266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:39.355266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:39.355266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:39.355266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:39.355266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:39.355266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:14:39.355266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:39.355266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 228/459] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:39.355832 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0783d0-8f74-414f-85da-c56cd5e4a298 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.380289 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314df8f-183b-4461-bce5-117ebc93959f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.380927 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314df8f-183b-4461-bce5-117ebc93959f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.381209 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314df8f-183b-4461-bce5-117ebc93959f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.381486 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314df8f-183b-4461-bce5-117ebc93959f None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.382123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314df8f-183b-4461-bce5-117ebc93959f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:39.382428 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314df8f-183b-4461-bce5-117ebc93959f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.382673 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:39.383895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314df8f-183b-4461-bce5-117ebc93959f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.396229 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 232/460] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:39.397035 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314df8f-183b-4461-bce5-117ebc93959f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.415050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:39.453773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0783d0-8f74-414f-85da-c56cd5e4a298 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.487037 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266f5b8-cee7-45f0-9c40-2d2fc31cb657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.487645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f5b8-cee7-45f0-9c40-2d2fc31cb657 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.487924 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f5b8-cee7-45f0-9c40-2d2fc31cb657 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.488198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f5b8-cee7-45f0-9c40-2d2fc31cb657 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.488954 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266f5b8-cee7-45f0-9c40-2d2fc31cb657 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:39.489266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266f5b8-cee7-45f0-9c40-2d2fc31cb657 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.490701 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266f5b8-cee7-45f0-9c40-2d2fc31cb657 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.495220 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314df8f-183b-4461-bce5-117ebc93959f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.503038 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 229/461] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:39.503836 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266f5b8-cee7-45f0-9c40-2d2fc31cb657 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.533327 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b91803-add9-4777-ada0-b01d2f621eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.534416 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b91803-add9-4777-ada0-b01d2f621eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.534692 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b91803-add9-4777-ada0-b01d2f621eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.534960 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b91803-add9-4777-ada0-b01d2f621eb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.535670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b91803-add9-4777-ada0-b01d2f621eb3 None admin] RBAC: Adding query filter params (role.id=d276003aacd44f11a8b5e733a9a54f02, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=00ab674b018e474ab363e8e17d225e27) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:39.535975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b91803-add9-4777-ada0-b01d2f621eb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.537425 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b91803-add9-4777-ada0-b01d2f621eb3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.542969 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 233/462] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:39 2022] GET /identity/v3/role_assignments?user.id=00ab674b018e474ab363e8e17d225e27&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=d276003aacd44f11a8b5e733a9a54f02 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:39.543737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b91803-add9-4777-ada0-b01d2f621eb3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.598707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:39.613297 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266f5b8-cee7-45f0-9c40-2d2fc31cb657 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.654532 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef44371-fc55-4eb1-85f6-b073b0d880a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.655066 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef44371-fc55-4eb1-85f6-b073b0d880a3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.655344 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef44371-fc55-4eb1-85f6-b073b0d880a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.655623 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef44371-fc55-4eb1-85f6-b073b0d880a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.656281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef44371-fc55-4eb1-85f6-b073b0d880a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:39.656611 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef44371-fc55-4eb1-85f6-b073b0d880a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.658014 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef44371-fc55-4eb1-85f6-b073b0d880a3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.659334 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:39.676327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 230/463] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:39.676327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef44371-fc55-4eb1-85f6-b073b0d880a3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.686438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b91803-add9-4777-ada0-b01d2f621eb3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.724892 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2299d-6b3b-44ef-afb2-3fdcac172bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.724892 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2299d-6b3b-44ef-afb2-3fdcac172bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.725153 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2299d-6b3b-44ef-afb2-3fdcac172bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.725334 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2299d-6b3b-44ef-afb2-3fdcac172bb9 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.726141 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2299d-6b3b-44ef-afb2-3fdcac172bb9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:39.726463 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2299d-6b3b-44ef-afb2-3fdcac172bb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.727896 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2299d-6b3b-44ef-afb2-3fdcac172bb9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.747704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 234/464] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Nov 2 07:14:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:39.748503 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2299d-6b3b-44ef-afb2-3fdcac172bb9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.783294 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef44371-fc55-4eb1-85f6-b073b0d880a3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.819057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc0cdb-c26c-4cbc-868d-f678ada1caa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.819799 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc0cdb-c26c-4cbc-868d-f678ada1caa3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.820080 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc0cdb-c26c-4cbc-868d-f678ada1caa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.820353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc0cdb-c26c-4cbc-868d-f678ada1caa3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.827982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc0cdb-c26c-4cbc-868d-f678ada1caa3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.829458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc0cdb-c26c-4cbc-868d-f678ada1caa3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bdfc0cdb-c26c-4cbc-868d-f678ada1caa3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:39.841558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:39.842459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:39.843346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:39.843346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:39.843346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:39.843346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:39.843346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:39.843346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:39.843346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:39.843346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:39.843346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:14:39.843346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:39.843346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 231/465] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:39.843897 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc0cdb-c26c-4cbc-868d-f678ada1caa3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.859176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:39.864425 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2299d-6b3b-44ef-afb2-3fdcac172bb9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:39.899829 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8142fd1-556f-4711-909a-01421338ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:39.900489 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142fd1-556f-4711-909a-01421338ac82 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.900823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142fd1-556f-4711-909a-01421338ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.901068 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142fd1-556f-4711-909a-01421338ac82 None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/7acc23ec5ccf4756a9e00403116ffe6f/roles/d276003aacd44f11a8b5e733a9a54f02` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.910497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:39.918687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8142fd1-556f-4711-909a-01421338ac82 None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=7acc23ec5ccf4756a9e00403116ffe6f, role_id=d276003aacd44f11a8b5e733a9a54f02)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:39.920070 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8142fd1-556f-4711-909a-01421338ac82 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:39.932890 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 235/466] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:39 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/7acc23ec5ccf4756a9e00403116ffe6f/roles/d276003aacd44f11a8b5e733a9a54f02 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:39.946434 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8142fd1-556f-4711-909a-01421338ac82 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:39.967744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc0cdb-c26c-4cbc-868d-f678ada1caa3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:40.011042 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b27407-ce38-44f0-bb75-a56a676cba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:40.011690 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b27407-ce38-44f0-bb75-a56a676cba6e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.012018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b27407-ce38-44f0-bb75-a56a676cba6e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.012398 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b27407-ce38-44f0-bb75-a56a676cba6e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.022002 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b27407-ce38-44f0-bb75-a56a676cba6e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:40.023969 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b27407-ce38-44f0-bb75-a56a676cba6e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-40b27407-ce38-44f0-bb75-a56a676cba6e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:40.037375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:40.038355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:40.039458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:40.039458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:40.039458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:40.039458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:40.039458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:40.039458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:40.039458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:40.039458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:40.039458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:14:40.039458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:40.039458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 232/467] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:40.040128 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b27407-ce38-44f0-bb75-a56a676cba6e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:40.098017 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8142fd1-556f-4711-909a-01421338ac82 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:40.122431 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.142109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ee9a-b0ba-46d6-87df-8e5ca16c7fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:40.142926 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ee9a-b0ba-46d6-87df-8e5ca16c7fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.143302 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ee9a-b0ba-46d6-87df-8e5ca16c7fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.143671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ee9a-b0ba-46d6-87df-8e5ca16c7fb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.168659 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b27407-ce38-44f0-bb75-a56a676cba6e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:40.202624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.218166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd7d7b-3008-4f89-ae19-b08e77965a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:40.218498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ee9a-b0ba-46d6-87df-8e5ca16c7fb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:40.219161 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd7d7b-3008-4f89-ae19-b08e77965a7a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.219584 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd7d7b-3008-4f89-ae19-b08e77965a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.220024 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd7d7b-3008-4f89-ae19-b08e77965a7a None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.220347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ee9a-b0ba-46d6-87df-8e5ca16c7fb1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:40.221618 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd7d7b-3008-4f89-ae19-b08e77965a7a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:40.222070 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd7d7b-3008-4f89-ae19-b08e77965a7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:40.223760 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd7d7b-3008-4f89-ae19-b08e77965a7a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:40.249631 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 233/468] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Nov 2 07:14:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:40.250776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd7d7b-3008-4f89-ae19-b08e77965a7a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:40.353537 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 236/469] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:40.354644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ee9a-b0ba-46d6-87df-8e5ca16c7fb1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:40.372969 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd7d7b-3008-4f89-ae19-b08e77965a7a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:40.403485 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.415898 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599126d-d91d-49db-a2fb-2c66d7866333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:40.417152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599126d-d91d-49db-a2fb-2c66d7866333 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.417579 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599126d-d91d-49db-a2fb-2c66d7866333 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.417990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599126d-d91d-49db-a2fb-2c66d7866333 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.418913 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599126d-d91d-49db-a2fb-2c66d7866333 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:40.419377 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599126d-d91d-49db-a2fb-2c66d7866333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:40.421807 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599126d-d91d-49db-a2fb-2c66d7866333 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:40.445234 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 234/470] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Nov 2 07:14:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:40.450605 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599126d-d91d-49db-a2fb-2c66d7866333 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:40.465948 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.482434 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ee9a-b0ba-46d6-87df-8e5ca16c7fb1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:40.520879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b5aec-aed2-4a09-93db-9e5c8a3b8cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:40.521708 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b5aec-aed2-4a09-93db-9e5c8a3b8cfb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.522134 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b5aec-aed2-4a09-93db-9e5c8a3b8cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.522539 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b5aec-aed2-4a09-93db-9e5c8a3b8cfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.554920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599126d-d91d-49db-a2fb-2c66d7866333 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:40.589923 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645df6d3-4be3-4e52-8045-3be17de03f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:40.590712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645df6d3-4be3-4e52-8045-3be17de03f76 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.591082 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645df6d3-4be3-4e52-8045-3be17de03f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.591450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645df6d3-4be3-4e52-8045-3be17de03f76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.597955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b5aec-aed2-4a09-93db-9e5c8a3b8cfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:40.599693 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b5aec-aed2-4a09-93db-9e5c8a3b8cfb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:40.603374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645df6d3-4be3-4e52-8045-3be17de03f76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:40.604852 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645df6d3-4be3-4e52-8045-3be17de03f76 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-645df6d3-4be3-4e52-8045-3be17de03f76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:40.621537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:40.622663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:40.623556 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:40.623556 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:40.623556 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:40.628336 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 235/471] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:40.628336 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645df6d3-4be3-4e52-8045-3be17de03f76 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:40.663600 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.707899 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.727751 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 237/472] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:40.729105 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b5aec-aed2-4a09-93db-9e5c8a3b8cfb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:40.738066 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645df6d3-4be3-4e52-8045-3be17de03f76 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:40.777573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5e407-2afa-4eed-a5ef-2826171a59ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:40.778445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e407-2afa-4eed-a5ef-2826171a59ff None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.778727 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e407-2afa-4eed-a5ef-2826171a59ff None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.779000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e407-2afa-4eed-a5ef-2826171a59ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.841874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b5aec-aed2-4a09-93db-9e5c8a3b8cfb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:40.859346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5e407-2afa-4eed-a5ef-2826171a59ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:40.860959 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5e407-2afa-4eed-a5ef-2826171a59ff None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:40.880107 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75ca9c-5b81-4923-bf0f-41257023eede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:40.881820 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75ca9c-5b81-4923-bf0f-41257023eede None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.882303 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75ca9c-5b81-4923-bf0f-41257023eede None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.882773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75ca9c-5b81-4923-bf0f-41257023eede None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.883773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75ca9c-5b81-4923-bf0f-41257023eede None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:40.884519 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75ca9c-5b81-4923-bf0f-41257023eede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:40.886107 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75ca9c-5b81-4923-bf0f-41257023eede None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:40.900831 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.902846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 238/473] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:40.903898 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75ca9c-5b81-4923-bf0f-41257023eede None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:40.975241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.988690 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 236/474] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:40.990582 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5e407-2afa-4eed-a5ef-2826171a59ff None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.044361 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75ca9c-5b81-4923-bf0f-41257023eede None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.088949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d46ef4-593f-4589-bcce-d16abc560125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.089878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46ef4-593f-4589-bcce-d16abc560125 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.090335 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46ef4-593f-4589-bcce-d16abc560125 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.091360 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46ef4-593f-4589-bcce-d16abc560125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.104329 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5e407-2afa-4eed-a5ef-2826171a59ff None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.106475 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d46ef4-593f-4589-bcce-d16abc560125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.109309 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d46ef4-593f-4589-bcce-d16abc560125 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c2d46ef4-593f-4589-bcce-d16abc560125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:41.128746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:41.129844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:41.130900 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:41.130900 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:41.130900 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:41.132666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 239/475] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:41.134491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d46ef4-593f-4589-bcce-d16abc560125 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.145957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3f2cf-f94d-4596-bc43-19acc6c52576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.146583 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f2cf-f94d-4596-bc43-19acc6c52576 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.147010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f2cf-f94d-4596-bc43-19acc6c52576 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.147373 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f2cf-f94d-4596-bc43-19acc6c52576 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.150943 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.156918 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad3f2cf-f94d-4596-bc43-19acc6c52576 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.159790 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad3f2cf-f94d-4596-bc43-19acc6c52576 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5ad3f2cf-f94d-4596-bc43-19acc6c52576 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:41.173221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:41.174379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:41.175441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:41.175441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:41.175441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:41.175441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:41.175441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:41.175441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:41.175441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:41.175441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:41.177438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 237/476] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:41.181346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3f2cf-f94d-4596-bc43-19acc6c52576 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.266086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.269746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d46ef4-593f-4589-bcce-d16abc560125 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.291175 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3f2cf-f94d-4596-bc43-19acc6c52576 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.309879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db9b18-6faf-43e7-aeba-b88564ccbd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.310880 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db9b18-6faf-43e7-aeba-b88564ccbd02 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.311302 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db9b18-6faf-43e7-aeba-b88564ccbd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.311745 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db9b18-6faf-43e7-aeba-b88564ccbd02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.323764 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db9b18-6faf-43e7-aeba-b88564ccbd02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.325459 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db9b18-6faf-43e7-aeba-b88564ccbd02 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.327904 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2e560-8c5d-4d89-b262-3154a0807411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.328570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2e560-8c5d-4d89-b262-3154a0807411 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.328833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2e560-8c5d-4d89-b262-3154a0807411 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.329101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2e560-8c5d-4d89-b262-3154a0807411 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.329736 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2e560-8c5d-4d89-b262-3154a0807411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:41.330038 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2e560-8c5d-4d89-b262-3154a0807411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.331947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2e560-8c5d-4d89-b262-3154a0807411 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-22db9b18-6faf-43e7-aeba-b88564ccbd02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:41.342760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:41.343846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:41.345028 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:41.345028 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:41.345028 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:41.346776 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 240/478] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:41.347863 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db9b18-6faf-43e7-aeba-b88564ccbd02 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.348384 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 238/478] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:41.349445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2e560-8c5d-4d89-b262-3154a0807411 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.406867 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.451831 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2e560-8c5d-4d89-b262-3154a0807411 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.461399 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db9b18-6faf-43e7-aeba-b88564ccbd02 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.487024 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f0e46-d9df-492d-94c6-746a35df61a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.487666 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f0e46-d9df-492d-94c6-746a35df61a0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.487942 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f0e46-d9df-492d-94c6-746a35df61a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.488213 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f0e46-d9df-492d-94c6-746a35df61a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.494969 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f0e46-d9df-492d-94c6-746a35df61a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.496505 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f0e46-d9df-492d-94c6-746a35df61a0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.500142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402b6ca-50aa-483a-bb16-01251d7acac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.500939 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402b6ca-50aa-483a-bb16-01251d7acac2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.501395 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402b6ca-50aa-483a-bb16-01251d7acac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.501807 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402b6ca-50aa-483a-bb16-01251d7acac2 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.502856 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402b6ca-50aa-483a-bb16-01251d7acac2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:41.503304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402b6ca-50aa-483a-bb16-01251d7acac2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.504879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402b6ca-50aa-483a-bb16-01251d7acac2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-370f0e46-d9df-492d-94c6-746a35df61a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:41.508349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:41.509318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:41.510232 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:41.510232 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:41.510232 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:41.510232 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:41.510232 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:41.510232 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:41.510232 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:41.510232 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:41.514252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 239/479] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:41.515073 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f0e46-d9df-492d-94c6-746a35df61a0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.521877 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 241/480] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:41.523085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402b6ca-50aa-483a-bb16-01251d7acac2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.538815 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.649840 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402b6ca-50aa-483a-bb16-01251d7acac2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.651945 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f0e46-d9df-492d-94c6-746a35df61a0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.670900 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.695800 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920e7e7-dd31-4151-8402-4ecdfac96893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.696247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1542b18-0c5d-4ba1-8fd3-76d3fe85d35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.696937 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920e7e7-dd31-4151-8402-4ecdfac96893 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.697043 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1542b18-0c5d-4ba1-8fd3-76d3fe85d35d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.697342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1542b18-0c5d-4ba1-8fd3-76d3fe85d35d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.697679 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1542b18-0c5d-4ba1-8fd3-76d3fe85d35d None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.697867 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920e7e7-dd31-4151-8402-4ecdfac96893 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.698296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920e7e7-dd31-4151-8402-4ecdfac96893 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.698598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1542b18-0c5d-4ba1-8fd3-76d3fe85d35d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:41.698964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1542b18-0c5d-4ba1-8fd3-76d3fe85d35d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.699538 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920e7e7-dd31-4151-8402-4ecdfac96893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:41.700023 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920e7e7-dd31-4151-8402-4ecdfac96893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.700623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1542b18-0c5d-4ba1-8fd3-76d3fe85d35d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.701744 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920e7e7-dd31-4151-8402-4ecdfac96893 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.717380 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 240/482] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:41.717585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 242/482] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:41.719951 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1542b18-0c5d-4ba1-8fd3-76d3fe85d35d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.722410 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920e7e7-dd31-4151-8402-4ecdfac96893 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.796656 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.830527 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1542b18-0c5d-4ba1-8fd3-76d3fe85d35d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.835367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920e7e7-dd31-4151-8402-4ecdfac96893 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.867813 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b1714-5072-4e25-9f10-2ec6081640bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.868425 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b1714-5072-4e25-9f10-2ec6081640bc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.868731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b1714-5072-4e25-9f10-2ec6081640bc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.869002 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b1714-5072-4e25-9f10-2ec6081640bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.869729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b1714-5072-4e25-9f10-2ec6081640bc None admin] RBAC: Adding query filter params (role.id=d276003aacd44f11a8b5e733a9a54f02, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=7a5b837bc9c04fc5a53077d152f4d6d5) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:41.870020 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b1714-5072-4e25-9f10-2ec6081640bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.871441 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b1714-5072-4e25-9f10-2ec6081640bc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.876467 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1fe1c-0178-44a7-9ecd-bf269862703f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.877180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 243/483] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/role_assignments?user.id=7a5b837bc9c04fc5a53077d152f4d6d5&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=d276003aacd44f11a8b5e733a9a54f02 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:41.877499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1fe1c-0178-44a7-9ecd-bf269862703f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.877897 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1fe1c-0178-44a7-9ecd-bf269862703f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.877992 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b1714-5072-4e25-9f10-2ec6081640bc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.878469 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1fe1c-0178-44a7-9ecd-bf269862703f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.879327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1fe1c-0178-44a7-9ecd-bf269862703f None admin] RBAC: Adding query filter params (role.id=d276003aacd44f11a8b5e733a9a54f02, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=36450acd551f40d08a4bc21c9a9c8877) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:41.879771 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1fe1c-0178-44a7-9ecd-bf269862703f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.882570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1fe1c-0178-44a7-9ecd-bf269862703f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.891620 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 241/484] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/role_assignments?user.id=36450acd551f40d08a4bc21c9a9c8877&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=d276003aacd44f11a8b5e733a9a54f02 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:41.893445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738526-653e-4a0b-82f6-e467d9119531 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.893893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738526-653e-4a0b-82f6-e467d9119531 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.894711 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738526-653e-4a0b-82f6-e467d9119531 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.896846 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 242/485] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:41.903192 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383c736-5535-4ad4-a723-55a71881ad34 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.903716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383c736-5535-4ad4-a723-55a71881ad34 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.904127 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383c736-5535-4ad4-a723-55a71881ad34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.931597 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9383c736-5535-4ad4-a723-55a71881ad34 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:41.933408 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9383c736-5535-4ad4-a723-55a71881ad34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:41.984611 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:42.020480 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b1714-5072-4e25-9f10-2ec6081640bc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:42.061349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dffdf-6674-47bd-b1d8-462c24e31667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:42.062335 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dffdf-6674-47bd-b1d8-462c24e31667 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.062762 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dffdf-6674-47bd-b1d8-462c24e31667 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.063306 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dffdf-6674-47bd-b1d8-462c24e31667 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.067962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:42.070331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 243/486] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:42.075794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24dffdf-6674-47bd-b1d8-462c24e31667 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:42.077662 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24dffdf-6674-47bd-b1d8-462c24e31667 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e24dffdf-6674-47bd-b1d8-462c24e31667 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:42.091484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:42.092606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:42.093644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:42.093644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:42.093644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:42.093644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:42.093644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:42.093644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:42.093644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:42.093644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:42.095903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 244/487] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:42.101655 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383c736-5535-4ad4-a723-55a71881ad34 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:42.210228 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef287cad-1f81-4056-9926-6aa4ed86c9ec None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.210741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef287cad-1f81-4056-9926-6aa4ed86c9ec None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.210741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef287cad-1f81-4056-9926-6aa4ed86c9ec None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.212471 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 245/488] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:42.219031 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa2c7-ee09-4928-879b-2ab06a7caf90 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.219808 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa2c7-ee09-4928-879b-2ab06a7caf90 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.220273 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa2c7-ee09-4928-879b-2ab06a7caf90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.227687 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383c736-5535-4ad4-a723-55a71881ad34 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:42.242956 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6c9fa2c7-ee09-4928-879b-2ab06a7caf90 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:42.244143 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6c9fa2c7-ee09-4928-879b-2ab06a7caf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:42.271800 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c4c4c-ce68-4ca2-a86d-10ade96ad55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:42.272414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c4c4c-ce68-4ca2-a86d-10ade96ad55c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.272749 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c4c4c-ce68-4ca2-a86d-10ade96ad55c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.273023 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c4c4c-ce68-4ca2-a86d-10ade96ad55c None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.273725 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c4c4c-ce68-4ca2-a86d-10ade96ad55c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:42.274029 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c4c4c-ce68-4ca2-a86d-10ade96ad55c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:42.276253 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c4c4c-ce68-4ca2-a86d-10ade96ad55c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:42.295054 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 244/489] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:42.296311 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c4c4c-ce68-4ca2-a86d-10ade96ad55c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:42.333119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 246/490] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:42.337296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:42.409663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c4c4c-ce68-4ca2-a86d-10ade96ad55c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:42.419434 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fa2c7-ee09-4928-879b-2ab06a7caf90 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:42.445605 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134b47-4262-4d05-9b89-b523c82ff8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:42.446215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134b47-4262-4d05-9b89-b523c82ff8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.446493 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134b47-4262-4d05-9b89-b523c82ff8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.446766 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134b47-4262-4d05-9b89-b523c82ff8a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.447493 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa134b47-4262-4d05-9b89-b523c82ff8a3 None admin] RBAC: Adding query filter params (role.id=d276003aacd44f11a8b5e733a9a54f02, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=e0f9d9dc1f7143e0be709e17d1a5f484) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:42.447802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa134b47-4262-4d05-9b89-b523c82ff8a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:42.449377 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa134b47-4262-4d05-9b89-b523c82ff8a3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:42.459640 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 245/491] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:42 2022] GET /identity/v3/role_assignments?user.id=e0f9d9dc1f7143e0be709e17d1a5f484&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=d276003aacd44f11a8b5e733a9a54f02 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:42.602323 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fa2c7-ee09-4928-879b-2ab06a7caf90 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:42.631026 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:42.650684 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb55ae-f2f9-469d-9b03-193d5464e451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:42.651437 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb55ae-f2f9-469d-9b03-193d5464e451 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.651685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb55ae-f2f9-469d-9b03-193d5464e451 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.651953 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb55ae-f2f9-469d-9b03-193d5464e451 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.668818 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb55ae-f2f9-469d-9b03-193d5464e451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0cdb55ae-f2f9-469d-9b03-193d5464e451 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:42.673660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:42.675094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:42.676456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 247/492] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:42.681796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134b47-4262-4d05-9b89-b523c82ff8a3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:42.742006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb55ae-f2f9-469d-9b03-193d5464e451 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:42.858918 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb55ae-f2f9-469d-9b03-193d5464e451 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:42.868098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:42.886704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134b47-4262-4d05-9b89-b523c82ff8a3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:42.907177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6a637-c076-4b91-97e7-0dca105c0d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:42.907821 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6a637-c076-4b91-97e7-0dca105c0d12 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.908096 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6a637-c076-4b91-97e7-0dca105c0d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.908415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6a637-c076-4b91-97e7-0dca105c0d12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.936102 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6a637-c076-4b91-97e7-0dca105c0d12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c7a6a637-c076-4b91-97e7-0dca105c0d12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:42.941006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:42.941878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:42.943169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 248/493] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:42.950645 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6a637-c076-4b91-97e7-0dca105c0d12 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:42.969281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646d49a-2b04-43f4-ba03-de60ac0ebedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:42.969909 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646d49a-2b04-43f4-ba03-de60ac0ebedf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.970191 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646d49a-2b04-43f4-ba03-de60ac0ebedf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.970468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646d49a-2b04-43f4-ba03-de60ac0ebedf None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.971705 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646d49a-2b04-43f4-ba03-de60ac0ebedf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:42.972005 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646d49a-2b04-43f4-ba03-de60ac0ebedf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:42.973406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646d49a-2b04-43f4-ba03-de60ac0ebedf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:42.985884 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 246/494] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:42.991179 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646d49a-2b04-43f4-ba03-de60ac0ebedf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:43.006815 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:43.087505 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6a637-c076-4b91-97e7-0dca105c0d12 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:43.131056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646d49a-2b04-43f4-ba03-de60ac0ebedf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:43.155411 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec97db-e17d-4f3c-a62b-b1141e315b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:43.156326 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec97db-e17d-4f3c-a62b-b1141e315b73 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.156796 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec97db-e17d-4f3c-a62b-b1141e315b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.157287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec97db-e17d-4f3c-a62b-b1141e315b73 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.158251 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec97db-e17d-4f3c-a62b-b1141e315b73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:43.159407 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec97db-e17d-4f3c-a62b-b1141e315b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:43.161087 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec97db-e17d-4f3c-a62b-b1141e315b73 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:43.177229 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dceba-ec09-4979-848e-d7102a8beb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:43.177875 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dceba-ec09-4979-848e-d7102a8beb65 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.178157 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dceba-ec09-4979-848e-d7102a8beb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.178436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dceba-ec09-4979-848e-d7102a8beb65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.180643 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 249/495] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:43.188685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec97db-e17d-4f3c-a62b-b1141e315b73 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:43.200844 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57dceba-ec09-4979-848e-d7102a8beb65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:43.202299 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57dceba-ec09-4979-848e-d7102a8beb65 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f57dceba-ec09-4979-848e-d7102a8beb65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:43.230041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:43.231026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:43.232071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:43.232071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:43.232071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:43.232071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 247/496] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:43.237978 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dceba-ec09-4979-848e-d7102a8beb65 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:43.265754 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:43.356011 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec97db-e17d-4f3c-a62b-b1141e315b73 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:43.368054 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:43.410634 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dceba-ec09-4979-848e-d7102a8beb65 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:43.412515 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f20750-dafa-4496-b097-4c22cba2b2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:43.413175 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f20750-dafa-4496-b097-4c22cba2b2fb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.413452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f20750-dafa-4496-b097-4c22cba2b2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.413724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f20750-dafa-4496-b097-4c22cba2b2fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.431881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f20750-dafa-4496-b097-4c22cba2b2fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:43.433364 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f20750-dafa-4496-b097-4c22cba2b2fb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-02f20750-dafa-4496-b097-4c22cba2b2fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:43.456162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:43.457142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:43.457999 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:43.457999 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:43.457999 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:43.458787 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 250/497] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:43.464821 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f20750-dafa-4496-b097-4c22cba2b2fb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:43.480392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086a1da-7b1e-4ff7-8235-3187c35bb461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:43.481027 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086a1da-7b1e-4ff7-8235-3187c35bb461 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.481309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086a1da-7b1e-4ff7-8235-3187c35bb461 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.481588 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086a1da-7b1e-4ff7-8235-3187c35bb461 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.482234 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086a1da-7b1e-4ff7-8235-3187c35bb461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:43.482538 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086a1da-7b1e-4ff7-8235-3187c35bb461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:43.483908 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086a1da-7b1e-4ff7-8235-3187c35bb461 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:43.497496 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 248/498] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:43.503099 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086a1da-7b1e-4ff7-8235-3187c35bb461 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:43.621861 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:43.670599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086a1da-7b1e-4ff7-8235-3187c35bb461 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:43.671208 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f20750-dafa-4496-b097-4c22cba2b2fb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:43.716830 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588d1ac-0168-4910-9ac6-2f11ea7b5904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:43.717455 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588d1ac-0168-4910-9ac6-2f11ea7b5904 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.717738 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588d1ac-0168-4910-9ac6-2f11ea7b5904 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.718000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588d1ac-0168-4910-9ac6-2f11ea7b5904 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.732619 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588d1ac-0168-4910-9ac6-2f11ea7b5904 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:43.734057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588d1ac-0168-4910-9ac6-2f11ea7b5904 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:43.746220 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d951b-874e-494d-94ef-de636e275fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:43.746829 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d951b-874e-494d-94ef-de636e275fec None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.747106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d951b-874e-494d-94ef-de636e275fec None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.747377 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d951b-874e-494d-94ef-de636e275fec None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.748011 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d951b-874e-494d-94ef-de636e275fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:43.748307 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d951b-874e-494d-94ef-de636e275fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:43.749947 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d951b-874e-494d-94ef-de636e275fec None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9588d1ac-0168-4910-9ac6-2f11ea7b5904 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:43.762437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:43.763327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:43.764773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:43.764773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:43.764773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:43.764773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:43.764773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:43.764773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:43.764773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:43.764773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:43.764773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:14:43.764773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:43.764773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 249/499] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Nov 2 07:14:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:43.769040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588d1ac-0168-4910-9ac6-2f11ea7b5904 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:43.770478 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 251/500] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:43.779610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d951b-874e-494d-94ef-de636e275fec None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:43.816292 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:43.928000 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d951b-874e-494d-94ef-de636e275fec None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:43.964962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56424634-08a9-4afd-b4fc-a33c56ecbc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:43.965594 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56424634-08a9-4afd-b4fc-a33c56ecbc6a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:43.965870 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56424634-08a9-4afd-b4fc-a33c56ecbc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:43.966140 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56424634-08a9-4afd-b4fc-a33c56ecbc6a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:43.973835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56424634-08a9-4afd-b4fc-a33c56ecbc6a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:43.974816 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588d1ac-0168-4910-9ac6-2f11ea7b5904 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:43.975272 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56424634-08a9-4afd-b4fc-a33c56ecbc6a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-56424634-08a9-4afd-b4fc-a33c56ecbc6a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:43.987451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:43.989688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:43.992050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:43.992050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:43.992050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:43.992050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:43.992050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:43.992050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:43.992050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:43.992050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:43.992050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:14:43.992050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:43.992050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 252/501] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:43.995380 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56424634-08a9-4afd-b4fc-a33c56ecbc6a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.031190 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.049282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc46aa3-dd6c-43ae-9455-687939a9e275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.050020 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc46aa3-dd6c-43ae-9455-687939a9e275 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.050219 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc46aa3-dd6c-43ae-9455-687939a9e275 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.050482 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc46aa3-dd6c-43ae-9455-687939a9e275 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.051229 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc46aa3-dd6c-43ae-9455-687939a9e275 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:44.051536 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc46aa3-dd6c-43ae-9455-687939a9e275 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:44.053001 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc46aa3-dd6c-43ae-9455-687939a9e275 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:44.088561 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 250/502] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Nov 2 07:14:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:44.091207 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.094203 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc46aa3-dd6c-43ae-9455-687939a9e275 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.113386 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56424634-08a9-4afd-b4fc-a33c56ecbc6a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.172961 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58549f45-156f-453b-b70d-1c89c23b6bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.173572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58549f45-156f-453b-b70d-1c89c23b6bee None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.173847 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58549f45-156f-453b-b70d-1c89c23b6bee None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.174720 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58549f45-156f-453b-b70d-1c89c23b6bee None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.175449 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58549f45-156f-453b-b70d-1c89c23b6bee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:44.175748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58549f45-156f-453b-b70d-1c89c23b6bee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:44.177266 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58549f45-156f-453b-b70d-1c89c23b6bee None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:44.207367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 253/503] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Nov 2 07:14:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:44.214964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58549f45-156f-453b-b70d-1c89c23b6bee None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.284443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc46aa3-dd6c-43ae-9455-687939a9e275 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.352697 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aa44b-2222-49c9-89fe-239c084e0b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.353334 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aa44b-2222-49c9-89fe-239c084e0b36 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.353610 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aa44b-2222-49c9-89fe-239c084e0b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.353880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aa44b-2222-49c9-89fe-239c084e0b36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.374454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73aa44b-2222-49c9-89fe-239c084e0b36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:44.376007 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73aa44b-2222-49c9-89fe-239c084e0b36 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d73aa44b-2222-49c9-89fe-239c084e0b36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:44.403443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:44.404504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:44.405406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:44.405406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:44.405406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:44.405998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58549f45-156f-453b-b70d-1c89c23b6bee None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.414435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 251/504] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:44.419555 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aa44b-2222-49c9-89fe-239c084e0b36 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.469240 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec900f65-0680-4241-ab03-8c23b1fc6fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.469771 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec900f65-0680-4241-ab03-8c23b1fc6fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.470049 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec900f65-0680-4241-ab03-8c23b1fc6fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.470316 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec900f65-0680-4241-ab03-8c23b1fc6fb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.484876 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.493806 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.565491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec900f65-0680-4241-ab03-8c23b1fc6fb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:44.567967 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec900f65-0680-4241-ab03-8c23b1fc6fb7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:44.572348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aa44b-2222-49c9-89fe-239c084e0b36 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.619330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa73eeb-2be0-4915-9ce3-740baa968fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.620031 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa73eeb-2be0-4915-9ce3-740baa968fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.620377 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa73eeb-2be0-4915-9ce3-740baa968fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.620954 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa73eeb-2be0-4915-9ce3-740baa968fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.621750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa73eeb-2be0-4915-9ce3-740baa968fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:44.622133 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa73eeb-2be0-4915-9ce3-740baa968fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:44.624318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa73eeb-2be0-4915-9ce3-740baa968fe2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:44.647718 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 252/505] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:44.652163 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1d233-f076-43a0-84f8-b4775188e8b5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.652163 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1d233-f076-43a0-84f8-b4775188e8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.652163 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1d233-f076-43a0-84f8-b4775188e8b5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.655080 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 253/506] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:44.656633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1d233-f076-43a0-84f8-b4775188e8b5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.696581 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 254/507] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:14:44.697986 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479151-e220-4d64-a243-47193c2e5c39 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.698300 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479151-e220-4d64-a243-47193c2e5c39 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.698638 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479151-e220-4d64-a243-47193c2e5c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.716556 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2d479151-e220-4d64-a243-47193c2e5c39 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:44.717755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2d479151-e220-4d64-a243-47193c2e5c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:44.758490 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.767530 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:44.772589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1d233-f076-43a0-84f8-b4775188e8b5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.806486 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 255/508] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:44.808045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31d5a7-d539-4982-8aa7-394ea17fd55e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.808329 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31d5a7-d539-4982-8aa7-394ea17fd55e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.808655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31d5a7-d539-4982-8aa7-394ea17fd55e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.810101 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 256/509] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:44.814965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31d5a7-d539-4982-8aa7-394ea17fd55e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.818953 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237cae5-7a9b-4455-b47b-1682caaa8311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.819863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237cae5-7a9b-4455-b47b-1682caaa8311 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.820364 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237cae5-7a9b-4455-b47b-1682caaa8311 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.820760 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237cae5-7a9b-4455-b47b-1682caaa8311 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.828660 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237cae5-7a9b-4455-b47b-1682caaa8311 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:44.830259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237cae5-7a9b-4455-b47b-1682caaa8311 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2237cae5-7a9b-4455-b47b-1682caaa8311 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:44.842333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:44.843215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:44.844083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:44.844083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:44.844083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:44.844083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:44.844083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:44.844083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:44.844083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:44.844083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:44.844083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 254/510] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:44.845543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a132e-1ebf-4e96-aa2c-f2ef52c8d9c6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.846419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a132e-1ebf-4e96-aa2c-f2ef52c8d9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.846771 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a132e-1ebf-4e96-aa2c-f2ef52c8d9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.865668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-352a132e-1ebf-4e96-aa2c-f2ef52c8d9c6 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:44.866947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-352a132e-1ebf-4e96-aa2c-f2ef52c8d9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:44.940314 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31d5a7-d539-4982-8aa7-394ea17fd55e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:44.948944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 255/511] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:44.950271 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a132e-1ebf-4e96-aa2c-f2ef52c8d9c6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:44.987095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c263d3-d3e8-4284-89a2-e4e4a9523f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:44.988017 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c263d3-d3e8-4284-89a2-e4e4a9523f74 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.988385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c263d3-d3e8-4284-89a2-e4e4a9523f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.988761 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c263d3-d3e8-4284-89a2-e4e4a9523f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.001256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c263d3-d3e8-4284-89a2-e4e4a9523f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.003264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c263d3-d3e8-4284-89a2-e4e4a9523f74 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-16c263d3-d3e8-4284-89a2-e4e4a9523f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:45.030689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:45.031938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:45.032869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:45.032869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:45.032869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:45.032869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 257/512] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:45.038891 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c263d3-d3e8-4284-89a2-e4e4a9523f74 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.078958 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.111254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.124248 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a132e-1ebf-4e96-aa2c-f2ef52c8d9c6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.161500 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c25e5-c694-4eaa-9e97-918ffd8c54ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.162198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c25e5-c694-4eaa-9e97-918ffd8c54ae None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.162542 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c25e5-c694-4eaa-9e97-918ffd8c54ae None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.162878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c25e5-c694-4eaa-9e97-918ffd8c54ae None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.163653 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c25e5-c694-4eaa-9e97-918ffd8c54ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:45.163840 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c263d3-d3e8-4284-89a2-e4e4a9523f74 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.164062 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c25e5-c694-4eaa-9e97-918ffd8c54ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.165586 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c25e5-c694-4eaa-9e97-918ffd8c54ae None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.177637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 256/513] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.183335 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9167e-0e73-4b62-81f2-e71c3890668c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.183619 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9167e-0e73-4b62-81f2-e71c3890668c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.183898 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9167e-0e73-4b62-81f2-e71c3890668c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.185297 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 257/514] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:45.185831 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9167e-0e73-4b62-81f2-e71c3890668c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.204215 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d27ada-db15-44bf-8efb-5885763bf659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.204975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d27ada-db15-44bf-8efb-5885763bf659 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.205331 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d27ada-db15-44bf-8efb-5885763bf659 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.205600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d27ada-db15-44bf-8efb-5885763bf659 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.206234 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d27ada-db15-44bf-8efb-5885763bf659 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:45.206532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d27ada-db15-44bf-8efb-5885763bf659 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.208004 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d27ada-db15-44bf-8efb-5885763bf659 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.225189 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 258/515] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:45.226024 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d27ada-db15-44bf-8efb-5885763bf659 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.288854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9167e-0e73-4b62-81f2-e71c3890668c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.306334 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.322577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371c49f-3841-4737-a1f3-dc2db1076f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.323202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371c49f-3841-4737-a1f3-dc2db1076f4e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.323481 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371c49f-3841-4737-a1f3-dc2db1076f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.323752 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371c49f-3841-4737-a1f3-dc2db1076f4e None admin] PATH_INFO: `/v3/services/network` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.324330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371c49f-3841-4737-a1f3-dc2db1076f4e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.325711 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371c49f-3841-4737-a1f3-dc2db1076f4e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.325804 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d27ada-db15-44bf-8efb-5885763bf659 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6371c49f-3841-4737-a1f3-dc2db1076f4e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:45.337235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:45.338275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:45.339183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 258/516] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/services/network => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:45.339986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219d8f0-ec94-47b2-a664-bd086c4cafd3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.339986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219d8f0-ec94-47b2-a664-bd086c4cafd3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.339986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219d8f0-ec94-47b2-a664-bd086c4cafd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.357236 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f219d8f0-ec94-47b2-a664-bd086c4cafd3 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:45.358419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f219d8f0-ec94-47b2-a664-bd086c4cafd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:45.363189 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6268d35-06b5-47f5-9322-54937bb3c7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.363840 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6268d35-06b5-47f5-9322-54937bb3c7f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.364116 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6268d35-06b5-47f5-9322-54937bb3c7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.364381 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6268d35-06b5-47f5-9322-54937bb3c7f5 None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/00ab674b018e474ab363e8e17d225e27/roles/81d287a8b606458d87c9607e8b48ac58` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.365562 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.366452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6268d35-06b5-47f5-9322-54937bb3c7f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=00ab674b018e474ab363e8e17d225e27, role_id=81d287a8b606458d87c9607e8b48ac58)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.367883 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6268d35-06b5-47f5-9322-54937bb3c7f5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.380731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 259/517] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:45 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/00ab674b018e474ab363e8e17d225e27/roles/81d287a8b606458d87c9607e8b48ac58 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:45.381523 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6268d35-06b5-47f5-9322-54937bb3c7f5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.434626 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 259/518] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:45.441186 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219d8f0-ec94-47b2-a664-bd086c4cafd3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.529087 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6268d35-06b5-47f5-9322-54937bb3c7f5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.568778 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219d8f0-ec94-47b2-a664-bd086c4cafd3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.573875 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df45ac1-3ad9-4085-8e53-db3054f72fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.574508 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df45ac1-3ad9-4085-8e53-db3054f72fcb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.574783 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df45ac1-3ad9-4085-8e53-db3054f72fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.575052 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df45ac1-3ad9-4085-8e53-db3054f72fcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.583235 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df45ac1-3ad9-4085-8e53-db3054f72fcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.584967 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df45ac1-3ad9-4085-8e53-db3054f72fcb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.589215 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6df45ac1-3ad9-4085-8e53-db3054f72fcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:45.601285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:45.602404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:45.603485 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:45.603485 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:45.603485 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:45.603485 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:45.603485 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:45.603485 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:45.603485 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:45.603485 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:45.604670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 260/519] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:45.605710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df45ac1-3ad9-4085-8e53-db3054f72fcb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.611795 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1975b-0d48-46dd-949c-e2f0033c43dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.612676 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1975b-0d48-46dd-949c-e2f0033c43dc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.613100 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1975b-0d48-46dd-949c-e2f0033c43dc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.613531 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1975b-0d48-46dd-949c-e2f0033c43dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.625515 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1975b-0d48-46dd-949c-e2f0033c43dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.627782 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1975b-0d48-46dd-949c-e2f0033c43dc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.629983 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-62f1975b-0d48-46dd-949c-e2f0033c43dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:45.649239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:45.650170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:45.651029 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:45.651029 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:45.651029 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:45.651029 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 260/520] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:45.651652 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1975b-0d48-46dd-949c-e2f0033c43dc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.733086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df45ac1-3ad9-4085-8e53-db3054f72fcb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.766358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1975b-0d48-46dd-949c-e2f0033c43dc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.767243 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c0c2d-c43c-436d-8275-428f1749654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.768373 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c0c2d-c43c-436d-8275-428f1749654a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.768684 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c0c2d-c43c-436d-8275-428f1749654a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.768954 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c0c2d-c43c-436d-8275-428f1749654a None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.769582 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c0c2d-c43c-436d-8275-428f1749654a None admin] RBAC: Adding query filter params (name=network) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:45.769881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c0c2d-c43c-436d-8275-428f1749654a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.771257 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c0c2d-c43c-436d-8275-428f1749654a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.775562 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 261/521] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.776345 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c0c2d-c43c-436d-8275-428f1749654a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.807499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc408b4-22d8-4ba6-ae6e-b454ba01ac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.808154 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc408b4-22d8-4ba6-ae6e-b454ba01ac7e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.808433 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc408b4-22d8-4ba6-ae6e-b454ba01ac7e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.808732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc408b4-22d8-4ba6-ae6e-b454ba01ac7e None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.809463 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc408b4-22d8-4ba6-ae6e-b454ba01ac7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:45.809769 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc408b4-22d8-4ba6-ae6e-b454ba01ac7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.811560 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc408b4-22d8-4ba6-ae6e-b454ba01ac7e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.830751 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 261/522] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.831723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc408b4-22d8-4ba6-ae6e-b454ba01ac7e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.841238 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.890120 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c0c2d-c43c-436d-8275-428f1749654a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.891220 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.926932 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76cbb3-c01b-49db-84a6-a5571d6872ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.927538 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76cbb3-c01b-49db-84a6-a5571d6872ff None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.927814 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76cbb3-c01b-49db-84a6-a5571d6872ff None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.928081 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76cbb3-c01b-49db-84a6-a5571d6872ff None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.928846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76cbb3-c01b-49db-84a6-a5571d6872ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:45.929146 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76cbb3-c01b-49db-84a6-a5571d6872ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.930614 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76cbb3-c01b-49db-84a6-a5571d6872ff None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.938499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc408b4-22d8-4ba6-ae6e-b454ba01ac7e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.947448 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 262/523] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.948186 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76cbb3-c01b-49db-84a6-a5571d6872ff None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.974766 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcf9aa-26bb-4f59-bed4-f563ee8af26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.975394 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcf9aa-26bb-4f59-bed4-f563ee8af26c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.975668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcf9aa-26bb-4f59-bed4-f563ee8af26c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.975935 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcf9aa-26bb-4f59-bed4-f563ee8af26c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.976511 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcf9aa-26bb-4f59-bed4-f563ee8af26c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.977882 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcf9aa-26bb-4f59-bed4-f563ee8af26c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-85dcf9aa-26bb-4f59-bed4-f563ee8af26c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:45.986637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:45.987617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:45.988445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 262/524] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:45.989399 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcf9aa-26bb-4f59-bed4-f563ee8af26c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:46.056238 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76cbb3-c01b-49db-84a6-a5571d6872ff None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:46.087292 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:46.100795 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcf9aa-26bb-4f59-bed4-f563ee8af26c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:46.100983 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879bfb3-6b0f-4164-8a58-24df8efc7ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:46.101473 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879bfb3-6b0f-4164-8a58-24df8efc7ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:46.101751 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879bfb3-6b0f-4164-8a58-24df8efc7ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:46.102018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879bfb3-6b0f-4164-8a58-24df8efc7ef8 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:46.102772 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879bfb3-6b0f-4164-8a58-24df8efc7ef8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:46.103172 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879bfb3-6b0f-4164-8a58-24df8efc7ef8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:46.104530 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879bfb3-6b0f-4164-8a58-24df8efc7ef8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:46.120583 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 263/525] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:46.120583 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879bfb3-6b0f-4164-8a58-24df8efc7ef8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:46.132400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:46.153481 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48237d86-3c5c-4f3c-b827-7e090474ebcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:46.154323 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48237d86-3c5c-4f3c-b827-7e090474ebcb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:46.154598 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48237d86-3c5c-4f3c-b827-7e090474ebcb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:46.154866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48237d86-3c5c-4f3c-b827-7e090474ebcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:46.155623 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48237d86-3c5c-4f3c-b827-7e090474ebcb None admin] RBAC: Adding query filter params (role.id=d276003aacd44f11a8b5e733a9a54f02, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=7acc23ec5ccf4756a9e00403116ffe6f) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:46.155916 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48237d86-3c5c-4f3c-b827-7e090474ebcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:46.157712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48237d86-3c5c-4f3c-b827-7e090474ebcb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:46.172602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 263/526] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/role_assignments?user.id=7acc23ec5ccf4756a9e00403116ffe6f&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=d276003aacd44f11a8b5e733a9a54f02 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:46.172602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48237d86-3c5c-4f3c-b827-7e090474ebcb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:46.294292 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879bfb3-6b0f-4164-8a58-24df8efc7ef8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:46.345307 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65bbf1-f5aa-4454-a558-3a8efe69f762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:46.346738 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65bbf1-f5aa-4454-a558-3a8efe69f762 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:46.347020 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65bbf1-f5aa-4454-a558-3a8efe69f762 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:46.347288 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65bbf1-f5aa-4454-a558-3a8efe69f762 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:46.347944 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65bbf1-f5aa-4454-a558-3a8efe69f762 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:46.349503 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65bbf1-f5aa-4454-a558-3a8efe69f762 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:46.372026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48237d86-3c5c-4f3c-b827-7e090474ebcb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:46.392860 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 264/527] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:46 2022] POST /identity/v3/users => generated 254 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:46.436266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7ba4b-0863-4ca3-81fa-af4f0f7a0931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:46.436931 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7ba4b-0863-4ca3-81fa-af4f0f7a0931 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:46.437214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7ba4b-0863-4ca3-81fa-af4f0f7a0931 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:46.437491 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7ba4b-0863-4ca3-81fa-af4f0f7a0931 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:46.438135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7ba4b-0863-4ca3-81fa-af4f0f7a0931 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:46.438436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7ba4b-0863-4ca3-81fa-af4f0f7a0931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:46.440365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7ba4b-0863-4ca3-81fa-af4f0f7a0931 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:46.450164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 264/528] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Nov 2 07:14:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:46.451632 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65bbf1-f5aa-4454-a558-3a8efe69f762 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:46.508857 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:46.577964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65bbf1-f5aa-4454-a558-3a8efe69f762 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:46.618433 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5171bb-1714-4b46-9ecb-7667781fd9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:46.618977 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5171bb-1714-4b46-9ecb-7667781fd9cc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:46.619264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5171bb-1714-4b46-9ecb-7667781fd9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:46.619643 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5171bb-1714-4b46-9ecb-7667781fd9cc None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:46.620223 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5171bb-1714-4b46-9ecb-7667781fd9cc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:46.620607 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5171bb-1714-4b46-9ecb-7667781fd9cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:46.621981 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5171bb-1714-4b46-9ecb-7667781fd9cc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:46.627814 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 265/529] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Nov 2 07:14:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:47.799611 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed544a-3696-448a-b875-ccf891d2783f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:47.800184 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed544a-3696-448a-b875-ccf891d2783f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:47.800184 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed544a-3696-448a-b875-ccf891d2783f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:47.801964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 265/530] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:47.807071 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a932fe-b58c-46f3-a14e-f939c93b76b7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:47.807469 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a932fe-b58c-46f3-a14e-f939c93b76b7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:47.807632 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a932fe-b58c-46f3-a14e-f939c93b76b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:47.837721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-57a932fe-b58c-46f3-a14e-f939c93b76b7 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:47.838914 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-57a932fe-b58c-46f3-a14e-f939c93b76b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:47.922442 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 266/531] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:48.260743 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed544a-3696-448a-b875-ccf891d2783f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:48.327775 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:48.398940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed544a-3696-448a-b875-ccf891d2783f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:48.433492 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631308b-e19e-4710-9ea7-4a8c4ce60e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:48.434132 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631308b-e19e-4710-9ea7-4a8c4ce60e13 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.434414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631308b-e19e-4710-9ea7-4a8c4ce60e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.434686 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631308b-e19e-4710-9ea7-4a8c4ce60e13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.453669 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631308b-e19e-4710-9ea7-4a8c4ce60e13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3631308b-e19e-4710-9ea7-4a8c4ce60e13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:48.459187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:48.460145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:48.461008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:48.466946 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 266/532] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:48.468175 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a932fe-b58c-46f3-a14e-f939c93b76b7 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:48.518882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:48.597322 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a932fe-b58c-46f3-a14e-f939c93b76b7 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:48.655096 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b536a3-2239-447b-94c8-c17d0b11ca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:48.655750 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b536a3-2239-447b-94c8-c17d0b11ca00 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.656032 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b536a3-2239-447b-94c8-c17d0b11ca00 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.656306 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b536a3-2239-447b-94c8-c17d0b11ca00 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.657026 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b536a3-2239-447b-94c8-c17d0b11ca00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:48.657332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b536a3-2239-447b-94c8-c17d0b11ca00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:48.659322 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b536a3-2239-447b-94c8-c17d0b11ca00 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:48.669957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 267/533] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:48.677789 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631308b-e19e-4710-9ea7-4a8c4ce60e13 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:48.790600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1c383-eb0e-4f85-8d42-a8cd15c09a7b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.790897 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1c383-eb0e-4f85-8d42-a8cd15c09a7b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.791081 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1c383-eb0e-4f85-8d42-a8cd15c09a7b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.792422 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 268/534] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:48.797905 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a5a5d-0ca1-4ee8-8b9b-cf5305203b09 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.798188 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a5a5d-0ca1-4ee8-8b9b-cf5305203b09 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.798531 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a5a5d-0ca1-4ee8-8b9b-cf5305203b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.825784 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6e7a5a5d-0ca1-4ee8-8b9b-cf5305203b09 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:48.826982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6e7a5a5d-0ca1-4ee8-8b9b-cf5305203b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:48.840528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631308b-e19e-4710-9ea7-4a8c4ce60e13 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:48.852866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:48.885064 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac24cef-35a7-407c-af82-7c0adc394fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:48.885986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac24cef-35a7-407c-af82-7c0adc394fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.886277 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac24cef-35a7-407c-af82-7c0adc394fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.886558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac24cef-35a7-407c-af82-7c0adc394fc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.899643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac24cef-35a7-407c-af82-7c0adc394fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:48.901868 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac24cef-35a7-407c-af82-7c0adc394fc9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4ac24cef-35a7-407c-af82-7c0adc394fc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:48.919947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:48.921156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:48.922252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:48.922252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:48.922252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:48.924626 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 267/535] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:48.926046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4148b9-04e2-4c37-b7c6-dc2fd3d92a39 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.926435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4148b9-04e2-4c37-b7c6-dc2fd3d92a39 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.926880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4148b9-04e2-4c37-b7c6-dc2fd3d92a39 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.928209 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 268/536] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:48.929180 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4148b9-04e2-4c37-b7c6-dc2fd3d92a39 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:48.938884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 269/537] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:48.940373 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f872f96-584b-409b-9f7b-a24b254d5ca1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.940690 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f872f96-584b-409b-9f7b-a24b254d5ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.940966 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f872f96-584b-409b-9f7b-a24b254d5ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.959549 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6f872f96-584b-409b-9f7b-a24b254d5ca1 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:48.960788 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6f872f96-584b-409b-9f7b-a24b254d5ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:49.003301 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.055368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4148b9-04e2-4c37-b7c6-dc2fd3d92a39 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:49.091132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 270/538] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:49.114527 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354a758-fa0d-483f-bef1-4d57a80ae082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.115141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a758-fa0d-483f-bef1-4d57a80ae082 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.115422 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a758-fa0d-483f-bef1-4d57a80ae082 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.115695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a758-fa0d-483f-bef1-4d57a80ae082 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.116348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354a758-fa0d-483f-bef1-4d57a80ae082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:49.116722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354a758-fa0d-483f-bef1-4d57a80ae082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:49.118156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354a758-fa0d-483f-bef1-4d57a80ae082 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:49.137782 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 269/539] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:49.143467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f872f96-584b-409b-9f7b-a24b254d5ca1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.289106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f872f96-584b-409b-9f7b-a24b254d5ca1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:49.327249 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238e4f-fda8-4166-af41-4557e6ce9799 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.327438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238e4f-fda8-4166-af41-4557e6ce9799 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.327716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238e4f-fda8-4166-af41-4557e6ce9799 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.330524 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 270/540] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:49.334548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d940be-2ade-422d-9498-8413ce146cc3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.334838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d940be-2ade-422d-9498-8413ce146cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.335112 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d940be-2ade-422d-9498-8413ce146cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.336563 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c76c69-be46-4d86-9515-53d113d48190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.337455 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c76c69-be46-4d86-9515-53d113d48190 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.337730 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c76c69-be46-4d86-9515-53d113d48190 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.338001 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c76c69-be46-4d86-9515-53d113d48190 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.351963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c76c69-be46-4d86-9515-53d113d48190 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:49.353424 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c76c69-be46-4d86-9515-53d113d48190 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:49.357703 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-11d940be-2ade-422d-9498-8413ce146cc3 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:49.359101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-11d940be-2ade-422d-9498-8413ce146cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f6c76c69-be46-4d86-9515-53d113d48190 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:49.372998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:49.373974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:49.374902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:49.374902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:49.374902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:49.374902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:49.374902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:49.374902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:49.374902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:49.374902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:49.374902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:14:49.374902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:49.377685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 271/541] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Nov 2 07:14:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:49.377793 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.378733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c76c69-be46-4d86-9515-53d113d48190 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.402372 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.443112 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 271/542] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:49.444093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d940be-2ade-422d-9498-8413ce146cc3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.495844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c76c69-be46-4d86-9515-53d113d48190 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:49.533368 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71ae0a-83fd-4380-aed9-507fde37fd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.534099 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71ae0a-83fd-4380-aed9-507fde37fd5d None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.534451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71ae0a-83fd-4380-aed9-507fde37fd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.534793 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71ae0a-83fd-4380-aed9-507fde37fd5d None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.535606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71ae0a-83fd-4380-aed9-507fde37fd5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:49.537167 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71ae0a-83fd-4380-aed9-507fde37fd5d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:49.557930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d940be-2ade-422d-9498-8413ce146cc3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:49.560676 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 272/543] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:14:49 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:49.561895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71ae0a-83fd-4380-aed9-507fde37fd5d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.646541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84f4bb-eeda-4373-9fed-cb04350eea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.647200 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84f4bb-eeda-4373-9fed-cb04350eea27 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.647485 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84f4bb-eeda-4373-9fed-cb04350eea27 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.647818 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84f4bb-eeda-4373-9fed-cb04350eea27 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.648976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84f4bb-eeda-4373-9fed-cb04350eea27 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:49.649290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84f4bb-eeda-4373-9fed-cb04350eea27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:49.650765 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84f4bb-eeda-4373-9fed-cb04350eea27 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:49.672812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 272/544] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Nov 2 07:14:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:49.675063 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afe0f1-fb0c-49c7-957d-e94a46e612d4 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.676250 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afe0f1-fb0c-49c7-957d-e94a46e612d4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.676573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afe0f1-fb0c-49c7-957d-e94a46e612d4 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.683377 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 273/545] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:49.684347 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afe0f1-fb0c-49c7-957d-e94a46e612d4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.688442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.692360 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.720585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71ae0a-83fd-4380-aed9-507fde37fd5d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:49.764278 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ab5af-e014-437c-b45f-45c0d24c8a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.764953 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab5af-e014-437c-b45f-45c0d24c8a0a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.765233 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab5af-e014-437c-b45f-45c0d24c8a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.765505 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab5af-e014-437c-b45f-45c0d24c8a0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.778160 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ab5af-e014-437c-b45f-45c0d24c8a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ec6ab5af-e014-437c-b45f-45c0d24c8a0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:49.785432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:49.786349 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:49.787161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:49.789667 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 273/546] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:49.791307 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8909-7d7a-44fd-abdc-754ff9f4b5a4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.791967 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8909-7d7a-44fd-abdc-754ff9f4b5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.792249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8909-7d7a-44fd-abdc-754ff9f4b5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.811062 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afe0f1-fb0c-49c7-957d-e94a46e612d4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:49.811253 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2f3b8909-7d7a-44fd-abdc-754ff9f4b5a4 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:49.812397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2f3b8909-7d7a-44fd-abdc-754ff9f4b5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:49.847985 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330494cc-bb04-45d2-a584-d98b103150fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:49.849338 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330494cc-bb04-45d2-a584-d98b103150fd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:49.849338 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330494cc-bb04-45d2-a584-d98b103150fd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:49.849338 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330494cc-bb04-45d2-a584-d98b103150fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:49.882266 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 274/547] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:49 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:49.893180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b8909-7d7a-44fd-abdc-754ff9f4b5a4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:49.939174 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330494cc-bb04-45d2-a584-d98b103150fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:49.940746 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330494cc-bb04-45d2-a584-d98b103150fd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:49.944890 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.953488 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:49.987611 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b8909-7d7a-44fd-abdc-754ff9f4b5a4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.022914 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eef2aa-9951-406f-8a16-a025b7450578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.023698 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef2aa-9951-406f-8a16-a025b7450578 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.024069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef2aa-9951-406f-8a16-a025b7450578 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.024436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef2aa-9951-406f-8a16-a025b7450578 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.035561 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eef2aa-9951-406f-8a16-a025b7450578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-99eef2aa-9951-406f-8a16-a025b7450578 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:50.040401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:50.041342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:50.042170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:50.046473 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 275/548] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:50.047385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eef2aa-9951-406f-8a16-a025b7450578 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.071784 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 274/549] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:49 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:50.073271 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330494cc-bb04-45d2-a584-d98b103150fd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.173702 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eef2aa-9951-406f-8a16-a025b7450578 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.188164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330494cc-bb04-45d2-a584-d98b103150fd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.190494 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.211401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.217984 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c877d-e6d5-4075-8826-8bd53f33c38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.218706 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c877d-e6d5-4075-8826-8bd53f33c38c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.219023 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c877d-e6d5-4075-8826-8bd53f33c38c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.219303 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c877d-e6d5-4075-8826-8bd53f33c38c None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.219957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c877d-e6d5-4075-8826-8bd53f33c38c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:50.220258 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c877d-e6d5-4075-8826-8bd53f33c38c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.222172 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c877d-e6d5-4075-8826-8bd53f33c38c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.230172 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 276/550] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:50.230951 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c877d-e6d5-4075-8826-8bd53f33c38c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.235247 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f0280-a961-49f4-8fcb-5f41d43250a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.236098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f0280-a961-49f4-8fcb-5f41d43250a2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.236784 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f0280-a961-49f4-8fcb-5f41d43250a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.237254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f0280-a961-49f4-8fcb-5f41d43250a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.238137 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f0280-a961-49f4-8fcb-5f41d43250a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:50.238631 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f0280-a961-49f4-8fcb-5f41d43250a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.240626 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f0280-a961-49f4-8fcb-5f41d43250a2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.251598 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 275/551] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:50.253329 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f0280-a961-49f4-8fcb-5f41d43250a2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.335368 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c877d-e6d5-4075-8826-8bd53f33c38c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.367433 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f0280-a961-49f4-8fcb-5f41d43250a2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.373549 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02517609-0264-40eb-8588-1a078df886dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.374177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02517609-0264-40eb-8588-1a078df886dd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.374863 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02517609-0264-40eb-8588-1a078df886dd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.375134 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02517609-0264-40eb-8588-1a078df886dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.387785 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02517609-0264-40eb-8588-1a078df886dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.389261 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02517609-0264-40eb-8588-1a078df886dd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.402816 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9566e0-a340-48b4-bf2c-ed9626d64d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.403422 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9566e0-a340-48b4-bf2c-ed9626d64d6b None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.403700 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9566e0-a340-48b4-bf2c-ed9626d64d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-02517609-0264-40eb-8588-1a078df886dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:50.403798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:50.404962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:50.405855 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:50.405855 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:50.405855 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:50.406029 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9566e0-a340-48b4-bf2c-ed9626d64d6b None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.406029 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9566e0-a340-48b4-bf2c-ed9626d64d6b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.406029 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9566e0-a340-48b4-bf2c-ed9626d64d6b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.409706 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 277/552] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:50.410229 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02517609-0264-40eb-8588-1a078df886dd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.422917 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 276/553] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:14:50 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:50.424740 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9566e0-a340-48b4-bf2c-ed9626d64d6b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.460081 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.464989 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.531396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02517609-0264-40eb-8588-1a078df886dd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.572778 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9566e0-a340-48b4-bf2c-ed9626d64d6b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.575393 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068e4e-ba46-4b07-a5da-9978c588ed9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.576147 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068e4e-ba46-4b07-a5da-9978c588ed9a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.577318 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068e4e-ba46-4b07-a5da-9978c588ed9a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.577318 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068e4e-ba46-4b07-a5da-9978c588ed9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.590613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3068e4e-ba46-4b07-a5da-9978c588ed9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.592388 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3068e4e-ba46-4b07-a5da-9978c588ed9a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b3068e4e-ba46-4b07-a5da-9978c588ed9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:50.608593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:50.609584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:50.610436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:50.610436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:50.610436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:50.610436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 278/554] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:50.611309 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068e4e-ba46-4b07-a5da-9978c588ed9a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.619226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e7e39-4a87-4f00-a6f4-5bf952c70344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.620147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e7e39-4a87-4f00-a6f4-5bf952c70344 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.620676 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e7e39-4a87-4f00-a6f4-5bf952c70344 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.621196 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e7e39-4a87-4f00-a6f4-5bf952c70344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.633076 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e7e39-4a87-4f00-a6f4-5bf952c70344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.634780 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e7e39-4a87-4f00-a6f4-5bf952c70344 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7d8e7e39-4a87-4f00-a6f4-5bf952c70344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:50.659773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:50.660796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:50.661689 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:50.661689 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:50.661689 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:50.662265 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 277/555] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:50.663317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e7e39-4a87-4f00-a6f4-5bf952c70344 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.723747 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.736473 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068e4e-ba46-4b07-a5da-9978c588ed9a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.760354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:50.786107 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d4970-1a38-4d79-adcd-4822263a5371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.787621 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d4970-1a38-4d79-adcd-4822263a5371 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.788047 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d4970-1a38-4d79-adcd-4822263a5371 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.788564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d4970-1a38-4d79-adcd-4822263a5371 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.790018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d4970-1a38-4d79-adcd-4822263a5371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:50.790471 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d4970-1a38-4d79-adcd-4822263a5371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.792621 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d4970-1a38-4d79-adcd-4822263a5371 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.796060 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e7e39-4a87-4f00-a6f4-5bf952c70344 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.809407 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 279/556] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:50.810730 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d4970-1a38-4d79-adcd-4822263a5371 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.833273 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79563d-38e1-45cd-a2eb-4f0f46d9b739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.834063 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79563d-38e1-45cd-a2eb-4f0f46d9b739 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.834440 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79563d-38e1-45cd-a2eb-4f0f46d9b739 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.834810 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79563d-38e1-45cd-a2eb-4f0f46d9b739 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.835557 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79563d-38e1-45cd-a2eb-4f0f46d9b739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:50.836300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79563d-38e1-45cd-a2eb-4f0f46d9b739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.837742 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79563d-38e1-45cd-a2eb-4f0f46d9b739 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.855359 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 278/557] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:50.855996 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79563d-38e1-45cd-a2eb-4f0f46d9b739 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.917236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d4970-1a38-4d79-adcd-4822263a5371 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.952770 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53baad-c16f-470a-945f-e497cbe0ff23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:50.953620 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53baad-c16f-470a-945f-e497cbe0ff23 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:50.954041 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53baad-c16f-470a-945f-e497cbe0ff23 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:50.954478 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53baad-c16f-470a-945f-e497cbe0ff23 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:50.955390 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53baad-c16f-470a-945f-e497cbe0ff23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:50.955834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53baad-c16f-470a-945f-e497cbe0ff23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:50.957353 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53baad-c16f-470a-945f-e497cbe0ff23 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:50.966732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79563d-38e1-45cd-a2eb-4f0f46d9b739 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:50.969983 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 280/558] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:50.971619 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53baad-c16f-470a-945f-e497cbe0ff23 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:50.987421 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.002146 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.008868 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4c38b-6fc3-4397-ada9-be53986b1b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.009504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4c38b-6fc3-4397-ada9-be53986b1b8b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.009782 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4c38b-6fc3-4397-ada9-be53986b1b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.010049 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4c38b-6fc3-4397-ada9-be53986b1b8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.018056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4c38b-6fc3-4397-ada9-be53986b1b8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.019561 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4c38b-6fc3-4397-ada9-be53986b1b8b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-74f4c38b-6fc3-4397-ada9-be53986b1b8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:51.033776 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:51.034691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:51.035646 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:51.035646 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:51.035646 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:51.035646 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:51.035646 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:51.035646 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:51.035646 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:51.035646 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:51.035646 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 279/559] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:51.036119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4c38b-6fc3-4397-ada9-be53986b1b8b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.083185 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53baad-c16f-470a-945f-e497cbe0ff23 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.118614 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebf65b-6e37-4bde-a55a-e7f50add213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.119537 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebf65b-6e37-4bde-a55a-e7f50add213c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.119999 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebf65b-6e37-4bde-a55a-e7f50add213c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.120465 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebf65b-6e37-4bde-a55a-e7f50add213c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.128422 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebf65b-6e37-4bde-a55a-e7f50add213c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.130082 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebf65b-6e37-4bde-a55a-e7f50add213c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6aebf65b-6e37-4bde-a55a-e7f50add213c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:51.141603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:51.143118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:51.144045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:51.144045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:51.144045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:51.144045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:51.144045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:51.144045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:51.144045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:51.144045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:51.144045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:14:51.144045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:51.145532 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4c38b-6fc3-4397-ada9-be53986b1b8b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.146363 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 281/560] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:51.147350 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebf65b-6e37-4bde-a55a-e7f50add213c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.182720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da96aa-cb8c-41da-b0ab-1432a695e7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.183489 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da96aa-cb8c-41da-b0ab-1432a695e7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.183770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da96aa-cb8c-41da-b0ab-1432a695e7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.184043 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da96aa-cb8c-41da-b0ab-1432a695e7b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.199462 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da96aa-cb8c-41da-b0ab-1432a695e7b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.201108 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da96aa-cb8c-41da-b0ab-1432a695e7b3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b9da96aa-cb8c-41da-b0ab-1432a695e7b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:51.214679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:51.215734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:51.216712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:51.216712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:51.216712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:51.216712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:51.216712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:51.216712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:51.216712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:51.216712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:51.216712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:14:51.216712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:51.220505 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 280/561] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:51.221607 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da96aa-cb8c-41da-b0ab-1432a695e7b3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.226701 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.260272 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebf65b-6e37-4bde-a55a-e7f50add213c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.283047 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.297696 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88057b26-1e68-460e-94fd-1cf590ac7d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.298491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88057b26-1e68-460e-94fd-1cf590ac7d03 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.298935 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88057b26-1e68-460e-94fd-1cf590ac7d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.299280 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88057b26-1e68-460e-94fd-1cf590ac7d03 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.300046 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88057b26-1e68-460e-94fd-1cf590ac7d03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:51.300423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88057b26-1e68-460e-94fd-1cf590ac7d03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.301960 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88057b26-1e68-460e-94fd-1cf590ac7d03 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.315485 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 282/562] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:51.316159 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88057b26-1e68-460e-94fd-1cf590ac7d03 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.342404 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da96aa-cb8c-41da-b0ab-1432a695e7b3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.379942 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25149f54-028e-4549-bf43-b4ef18b1830e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.381003 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25149f54-028e-4549-bf43-b4ef18b1830e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.381504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25149f54-028e-4549-bf43-b4ef18b1830e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.381953 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25149f54-028e-4549-bf43-b4ef18b1830e None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.382863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25149f54-028e-4549-bf43-b4ef18b1830e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:51.383348 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25149f54-028e-4549-bf43-b4ef18b1830e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.385228 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25149f54-028e-4549-bf43-b4ef18b1830e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.412023 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 281/563] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:51.413441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25149f54-028e-4549-bf43-b4ef18b1830e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.423472 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88057b26-1e68-460e-94fd-1cf590ac7d03 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.454265 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.460456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9867f4-e3ec-46bb-9bcc-ca65872f9d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.461890 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9867f4-e3ec-46bb-9bcc-ca65872f9d7b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.462376 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9867f4-e3ec-46bb-9bcc-ca65872f9d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.463097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9867f4-e3ec-46bb-9bcc-ca65872f9d7b None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.463773 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9867f4-e3ec-46bb-9bcc-ca65872f9d7b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:51.464184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9867f4-e3ec-46bb-9bcc-ca65872f9d7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.465793 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9867f4-e3ec-46bb-9bcc-ca65872f9d7b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.484499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 283/564] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:51.485715 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9867f4-e3ec-46bb-9bcc-ca65872f9d7b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.527353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25149f54-028e-4549-bf43-b4ef18b1830e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.539323 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.565688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae88fbc-c52f-42cf-af4b-30dd781240af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.566296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88fbc-c52f-42cf-af4b-30dd781240af None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.566574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88fbc-c52f-42cf-af4b-30dd781240af None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.566845 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88fbc-c52f-42cf-af4b-30dd781240af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.567677 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae88fbc-c52f-42cf-af4b-30dd781240af None admin] RBAC: Adding query filter params (role.id=81d287a8b606458d87c9607e8b48ac58, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=00ab674b018e474ab363e8e17d225e27) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:51.567989 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae88fbc-c52f-42cf-af4b-30dd781240af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.569539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae88fbc-c52f-42cf-af4b-30dd781240af None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.575920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 282/565] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/role_assignments?user.id=00ab674b018e474ab363e8e17d225e27&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=81d287a8b606458d87c9607e8b48ac58 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:51.576880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae88fbc-c52f-42cf-af4b-30dd781240af None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.602605 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9867f4-e3ec-46bb-9bcc-ca65872f9d7b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.667531 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe97523-d4aa-40b2-8f40-bc57784ea952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.668462 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe97523-d4aa-40b2-8f40-bc57784ea952 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.669096 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe97523-d4aa-40b2-8f40-bc57784ea952 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.669525 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe97523-d4aa-40b2-8f40-bc57784ea952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.693359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe97523-d4aa-40b2-8f40-bc57784ea952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.695078 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe97523-d4aa-40b2-8f40-bc57784ea952 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-afe97523-d4aa-40b2-8f40-bc57784ea952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:51.724964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:51.726193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:51.727280 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:51.727280 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:51.727280 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:51.730842 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 284/566] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:51.737114 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe97523-d4aa-40b2-8f40-bc57784ea952 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.764254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae88fbc-c52f-42cf-af4b-30dd781240af None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.791378 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.821981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1d98e-c9af-4eba-935f-081d22749dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.822620 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1d98e-c9af-4eba-935f-081d22749dbd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.822900 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1d98e-c9af-4eba-935f-081d22749dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.823173 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1d98e-c9af-4eba-935f-081d22749dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.836995 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1d98e-c9af-4eba-935f-081d22749dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.838405 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1d98e-c9af-4eba-935f-081d22749dbd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c4d1d98e-c9af-4eba-935f-081d22749dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:51.855894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:51.857400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:51.858301 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:51.858301 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:51.858301 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:51.858301 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 283/567] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:51.863438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1d98e-c9af-4eba-935f-081d22749dbd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:51.870569 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe97523-d4aa-40b2-8f40-bc57784ea952 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:51.899682 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.913016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f36a2-5925-4f1f-b113-d0457e860110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:51.913634 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f36a2-5925-4f1f-b113-d0457e860110 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.913914 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f36a2-5925-4f1f-b113-d0457e860110 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.914185 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f36a2-5925-4f1f-b113-d0457e860110 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.914832 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f36a2-5925-4f1f-b113-d0457e860110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:51.915135 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f36a2-5925-4f1f-b113-d0457e860110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:51.916506 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f36a2-5925-4f1f-b113-d0457e860110 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:51.931818 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 285/568] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:51.933379 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c00da-1082-4d9c-b448-57ef52611a00 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.933667 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c00da-1082-4d9c-b448-57ef52611a00 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.933938 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c00da-1082-4d9c-b448-57ef52611a00 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.935264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 286/569] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:51.939970 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c00da-1082-4d9c-b448-57ef52611a00 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.023821 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1d98e-c9af-4eba-935f-081d22749dbd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.051600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c00da-1082-4d9c-b448-57ef52611a00 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.053350 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.076507 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8a73c-a06f-4540-8a11-7170996fa751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.077761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8a73c-a06f-4540-8a11-7170996fa751 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.078666 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8a73c-a06f-4540-8a11-7170996fa751 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.079422 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8a73c-a06f-4540-8a11-7170996fa751 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.080606 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8a73c-a06f-4540-8a11-7170996fa751 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:52.081118 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8a73c-a06f-4540-8a11-7170996fa751 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.083383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8a73c-a06f-4540-8a11-7170996fa751 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.089802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7aff26-f9da-429c-989f-995c58d2d2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.090565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aff26-f9da-429c-989f-995c58d2d2ab None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.090912 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aff26-f9da-429c-989f-995c58d2d2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.091300 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aff26-f9da-429c-989f-995c58d2d2ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.099301 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7aff26-f9da-429c-989f-995c58d2d2ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.100957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7aff26-f9da-429c-989f-995c58d2d2ab None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.101175 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 284/570] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:52.102769 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d82745-af1f-417c-abcb-e518ad25b6e3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.103507 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d82745-af1f-417c-abcb-e518ad25b6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.104371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d82745-af1f-417c-abcb-e518ad25b6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4d7aff26-f9da-429c-989f-995c58d2d2ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:52.114231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:52.115244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:52.116132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:52.116132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:52.116132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:52.116132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:52.116132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:52.116132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:52.116132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:52.116132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:52.116132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 287/571] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:52.120746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7aff26-f9da-429c-989f-995c58d2d2ab None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.131342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-28d82745-af1f-417c-abcb-e518ad25b6e3 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:52.132819 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-28d82745-af1f-417c-abcb-e518ad25b6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:52.237553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7aff26-f9da-429c-989f-995c58d2d2ab None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.243778 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 285/572] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:14:52.248489 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d82745-af1f-417c-abcb-e518ad25b6e3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.284913 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.303546 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd26a2-ede7-4cfc-8cb1-01629f742d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.304113 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd26a2-ede7-4cfc-8cb1-01629f742d1a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.304390 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd26a2-ede7-4cfc-8cb1-01629f742d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.305281 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd26a2-ede7-4cfc-8cb1-01629f742d1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.320190 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd26a2-ede7-4cfc-8cb1-01629f742d1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.323567 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd26a2-ede7-4cfc-8cb1-01629f742d1a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-51fd26a2-ede7-4cfc-8cb1-01629f742d1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:52.346406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:52.348180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:52.349444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:52.349444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:52.349444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:52.349444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:52.349444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:52.349444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:52.349444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:52.349444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:52.351601 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 288/573] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:52.355001 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd26a2-ede7-4cfc-8cb1-01629f742d1a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.383213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.469719 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd26a2-ede7-4cfc-8cb1-01629f742d1a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.470057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d82745-af1f-417c-abcb-e518ad25b6e3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.510477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b3913-47bf-43bf-8692-0b9ff5c2337b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.511119 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b3913-47bf-43bf-8692-0b9ff5c2337b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.511396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b3913-47bf-43bf-8692-0b9ff5c2337b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.511670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b3913-47bf-43bf-8692-0b9ff5c2337b None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.512359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b3913-47bf-43bf-8692-0b9ff5c2337b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:52.512765 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b3913-47bf-43bf-8692-0b9ff5c2337b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.514529 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b3913-47bf-43bf-8692-0b9ff5c2337b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.531255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 289/574] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:52.537563 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b3913-47bf-43bf-8692-0b9ff5c2337b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.609394 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfeb27-d920-4439-bdd0-f6e2b10f092c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.609921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfeb27-d920-4439-bdd0-f6e2b10f092c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.610203 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfeb27-d920-4439-bdd0-f6e2b10f092c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.610476 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfeb27-d920-4439-bdd0-f6e2b10f092c None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.611185 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfeb27-d920-4439-bdd0-f6e2b10f092c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:52.612007 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfeb27-d920-4439-bdd0-f6e2b10f092c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.613492 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfeb27-d920-4439-bdd0-f6e2b10f092c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.627020 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.629525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 286/575] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:52.630752 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfeb27-d920-4439-bdd0-f6e2b10f092c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.657511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b3913-47bf-43bf-8692-0b9ff5c2337b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.698667 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08f00b-b31b-497b-a54c-5a95eb115a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.699364 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08f00b-b31b-497b-a54c-5a95eb115a07 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.699749 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08f00b-b31b-497b-a54c-5a95eb115a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.700031 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08f00b-b31b-497b-a54c-5a95eb115a07 None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/b67332ed32044f94a1b3a6a6e6bbb348/roles/d276003aacd44f11a8b5e733a9a54f02` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.720506 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08f00b-b31b-497b-a54c-5a95eb115a07 None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=b67332ed32044f94a1b3a6a6e6bbb348, role_id=d276003aacd44f11a8b5e733a9a54f02)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.722587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08f00b-b31b-497b-a54c-5a95eb115a07 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.733572 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.744654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 290/576] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:52 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/b67332ed32044f94a1b3a6a6e6bbb348/roles/d276003aacd44f11a8b5e733a9a54f02 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:14:52.749675 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08f00b-b31b-497b-a54c-5a95eb115a07 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.762903 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfeb27-d920-4439-bdd0-f6e2b10f092c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.809533 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd8560-b484-4692-8858-43a185fa28d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.810291 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd8560-b484-4692-8858-43a185fa28d4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.810572 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd8560-b484-4692-8858-43a185fa28d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.810869 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd8560-b484-4692-8858-43a185fa28d4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.811466 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd8560-b484-4692-8858-43a185fa28d4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.812908 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd8560-b484-4692-8858-43a185fa28d4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1fbd8560-b484-4692-8858-43a185fa28d4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:52.822707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:52.823930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:52.824811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 287/577] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:52.825647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f12c1c-04bf-4c91-8ed9-e541b32f63da None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.825647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f12c1c-04bf-4c91-8ed9-e541b32f63da None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.825647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f12c1c-04bf-4c91-8ed9-e541b32f63da None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.836484 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 288/578] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:52.836484 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f12c1c-04bf-4c91-8ed9-e541b32f63da None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.917389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08f00b-b31b-497b-a54c-5a95eb115a07 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.954736 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.963507 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f12c1c-04bf-4c91-8ed9-e541b32f63da None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.965771 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84218f-7e7f-4247-971f-266d5516a893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.966646 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84218f-7e7f-4247-971f-266d5516a893 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.967169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84218f-7e7f-4247-971f-266d5516a893 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.967656 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84218f-7e7f-4247-971f-266d5516a893 None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/7acc23ec5ccf4756a9e00403116ffe6f/roles/81d287a8b606458d87c9607e8b48ac58` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.970647 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84218f-7e7f-4247-971f-266d5516a893 None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=7acc23ec5ccf4756a9e00403116ffe6f, role_id=81d287a8b606458d87c9607e8b48ac58)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.972411 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84218f-7e7f-4247-971f-266d5516a893 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.994178 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 291/579] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:52 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/7acc23ec5ccf4756a9e00403116ffe6f/roles/81d287a8b606458d87c9607e8b48ac58 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:52.995587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aac2f-ad86-4aec-a6de-6e0166e4ef39 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.995871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aac2f-ad86-4aec-a6de-6e0166e4ef39 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.996148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aac2f-ad86-4aec-a6de-6e0166e4ef39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.010824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca91743-45b5-42b4-a6e8-d9b0631329d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.011865 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca91743-45b5-42b4-a6e8-d9b0631329d6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.012346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca91743-45b5-42b4-a6e8-d9b0631329d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.014101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca91743-45b5-42b4-a6e8-d9b0631329d6 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.014101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca91743-45b5-42b4-a6e8-d9b0631329d6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:53.014101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca91743-45b5-42b4-a6e8-d9b0631329d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.015148 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca91743-45b5-42b4-a6e8-d9b0631329d6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.021923 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0f6aac2f-ad86-4aec-a6de-6e0166e4ef39 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:53.026859 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 289/580] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.027086 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0f6aac2f-ad86-4aec-a6de-6e0166e4ef39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:53.029381 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca91743-45b5-42b4-a6e8-d9b0631329d6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.044718 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.152869 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca91743-45b5-42b4-a6e8-d9b0631329d6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.197612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb23e9-3b65-46a0-bac5-9bf47a187dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.198155 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb23e9-3b65-46a0-bac5-9bf47a187dde None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.198432 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb23e9-3b65-46a0-bac5-9bf47a187dde None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.198878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 292/581] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:14:53.198997 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb23e9-3b65-46a0-bac5-9bf47a187dde None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.199331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb23e9-3b65-46a0-bac5-9bf47a187dde None admin] RBAC: Adding query filter params (type=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:53.199629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb23e9-3b65-46a0-bac5-9bf47a187dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.202753 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb23e9-3b65-46a0-bac5-9bf47a187dde None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.207715 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 290/582] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.213497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aac2f-ad86-4aec-a6de-6e0166e4ef39 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.321616 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aac2f-ad86-4aec-a6de-6e0166e4ef39 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.346670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.359347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814b66a-b7be-4b3f-9c53-78679c07ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.360224 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814b66a-b7be-4b3f-9c53-78679c07ec99 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.360820 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814b66a-b7be-4b3f-9c53-78679c07ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.361859 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814b66a-b7be-4b3f-9c53-78679c07ec99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.362933 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814b66a-b7be-4b3f-9c53-78679c07ec99 None admin] RBAC: Adding query filter params (interface=public, service_id=db69f1415ee4489abf958a2f98b28649, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:53.363243 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814b66a-b7be-4b3f-9c53-78679c07ec99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.364669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814b66a-b7be-4b3f-9c53-78679c07ec99 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.376688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 293/583] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/endpoints?service_id=db69f1415ee4489abf958a2f98b28649&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.378431 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb23e9-3b65-46a0-bac5-9bf47a187dde None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.447134 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.483468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb23e9-3b65-46a0-bac5-9bf47a187dde None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.518124 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506022d6-7931-437b-a83a-623f607397b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.518743 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506022d6-7931-437b-a83a-623f607397b0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.519022 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506022d6-7931-437b-a83a-623f607397b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.519294 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506022d6-7931-437b-a83a-623f607397b0 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.519887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506022d6-7931-437b-a83a-623f607397b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.521287 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506022d6-7931-437b-a83a-623f607397b0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.532724 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 291/584] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/services => generated 772 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.595263 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814b66a-b7be-4b3f-9c53-78679c07ec99 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.748359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814b66a-b7be-4b3f-9c53-78679c07ec99 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.784727 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12df3f2-f0cd-4a7d-9ecd-3858a170e7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.785415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12df3f2-f0cd-4a7d-9ecd-3858a170e7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.785693 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12df3f2-f0cd-4a7d-9ecd-3858a170e7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.785963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12df3f2-f0cd-4a7d-9ecd-3858a170e7d2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.786547 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12df3f2-f0cd-4a7d-9ecd-3858a170e7d2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.787915 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12df3f2-f0cd-4a7d-9ecd-3858a170e7d2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e12df3f2-f0cd-4a7d-9ecd-3858a170e7d2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:53.800161 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:53.801279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:53.802196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 294/585] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:53.807115 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506022d6-7931-437b-a83a-623f607397b0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.915760 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506022d6-7931-437b-a83a-623f607397b0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.946195 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.951434 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5a299-8d68-47bb-86f6-04ae42bd1d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.952100 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5a299-8d68-47bb-86f6-04ae42bd1d66 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.952504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5a299-8d68-47bb-86f6-04ae42bd1d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.952810 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5a299-8d68-47bb-86f6-04ae42bd1d66 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.953443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5a299-8d68-47bb-86f6-04ae42bd1d66 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:53.953847 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5a299-8d68-47bb-86f6-04ae42bd1d66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.955307 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5a299-8d68-47bb-86f6-04ae42bd1d66 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.960428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 292/586] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:53.969644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12df3f2-f0cd-4a7d-9ecd-3858a170e7d2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:54.035477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:54.101595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12df3f2-f0cd-4a7d-9ecd-3858a170e7d2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:54.147570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f49f76-b8b5-4a6f-8433-393cd7b89d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:54.148191 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f49f76-b8b5-4a6f-8433-393cd7b89d67 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:54.148470 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f49f76-b8b5-4a6f-8433-393cd7b89d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:54.148775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f49f76-b8b5-4a6f-8433-393cd7b89d67 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:54.149407 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f49f76-b8b5-4a6f-8433-393cd7b89d67 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:54.149708 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f49f76-b8b5-4a6f-8433-393cd7b89d67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:54.151081 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f49f76-b8b5-4a6f-8433-393cd7b89d67 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:54.160959 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 295/587] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:54.166781 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5a299-8d68-47bb-86f6-04ae42bd1d66 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:54.316911 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5a299-8d68-47bb-86f6-04ae42bd1d66 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:54.351711 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca71c1f-e763-45d3-ac2c-b228b46cef17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:54.352709 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca71c1f-e763-45d3-ac2c-b228b46cef17 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:54.352995 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca71c1f-e763-45d3-ac2c-b228b46cef17 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:54.353266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca71c1f-e763-45d3-ac2c-b228b46cef17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:54.353960 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca71c1f-e763-45d3-ac2c-b228b46cef17 None admin] RBAC: Adding query filter params (interface=public, service_id=d6a7e1649db44e96bf4980b658376107, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:54.354267 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca71c1f-e763-45d3-ac2c-b228b46cef17 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:54.355677 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca71c1f-e763-45d3-ac2c-b228b46cef17 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:54.363707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 293/588] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Nov 2 07:14:54 2022] GET /identity/v3/endpoints?service_id=d6a7e1649db44e96bf4980b658376107&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:54.368875 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f49f76-b8b5-4a6f-8433-393cd7b89d67 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:54.462340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8565e25-65b0-4b9d-95a1-81661653fa7c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:54.462686 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8565e25-65b0-4b9d-95a1-81661653fa7c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:54.464224 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8565e25-65b0-4b9d-95a1-81661653fa7c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:54.468841 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 294/589] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:54.470623 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388c9f1-2e70-4fb4-858b-99902c47c595 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:54.470910 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388c9f1-2e70-4fb4-858b-99902c47c595 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:54.471183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388c9f1-2e70-4fb4-858b-99902c47c595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:54.480597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f49f76-b8b5-4a6f-8433-393cd7b89d67 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:54.505807 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3388c9f1-2e70-4fb4-858b-99902c47c595 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:54.507726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3388c9f1-2e70-4fb4-858b-99902c47c595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:54.515654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:54.550223 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc2bc6-f967-4921-bace-6793c7ab99eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:54.551246 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc2bc6-f967-4921-bace-6793c7ab99eb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:54.551720 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc2bc6-f967-4921-bace-6793c7ab99eb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:54.552184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc2bc6-f967-4921-bace-6793c7ab99eb None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:54.553101 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc2bc6-f967-4921-bace-6793c7ab99eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:54.554681 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc2bc6-f967-4921-bace-6793c7ab99eb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:54.572934 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 296/590] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Nov 2 07:14:54 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:54.596961 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:54.612642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 295/591] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:14:54.942256 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc2bc6-f967-4921-bace-6793c7ab99eb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:55.062730 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc2bc6-f967-4921-bace-6793c7ab99eb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:55.105241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70534e1e-4615-476f-9b88-db9a93bb9ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:55.105889 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70534e1e-4615-476f-9b88-db9a93bb9ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.106170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70534e1e-4615-476f-9b88-db9a93bb9ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.106442 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70534e1e-4615-476f-9b88-db9a93bb9ae4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.107019 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70534e1e-4615-476f-9b88-db9a93bb9ae4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:55.108396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70534e1e-4615-476f-9b88-db9a93bb9ae4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-70534e1e-4615-476f-9b88-db9a93bb9ae4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:55.117543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:55.118550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:55.119497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 297/592] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Nov 2 07:14:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:55.124602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388c9f1-2e70-4fb4-858b-99902c47c595 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:55.290291 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388c9f1-2e70-4fb4-858b-99902c47c595 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:55.294907 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633eedb-d23f-4e82-8e42-197e210fcd8b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.295178 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633eedb-d23f-4e82-8e42-197e210fcd8b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.295452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633eedb-d23f-4e82-8e42-197e210fcd8b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.297091 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 298/593] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:55.303913 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01132b97-d66b-4bd6-88f0-bef637a4ded1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.304180 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01132b97-d66b-4bd6-88f0-bef637a4ded1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.304456 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01132b97-d66b-4bd6-88f0-bef637a4ded1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.331059 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f0100-c7ea-4336-a005-6232f4307dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:55.331685 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f0100-c7ea-4336-a005-6232f4307dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.331969 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f0100-c7ea-4336-a005-6232f4307dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.332249 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f0100-c7ea-4336-a005-6232f4307dd6 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.333052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f0100-c7ea-4336-a005-6232f4307dd6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:55.333366 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f0100-c7ea-4336-a005-6232f4307dd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:55.334747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f0100-c7ea-4336-a005-6232f4307dd6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:55.343116 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 296/594] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Nov 2 07:14:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:55.344118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-01132b97-d66b-4bd6-88f0-bef637a4ded1 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:55.345356 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-01132b97-d66b-4bd6-88f0-bef637a4ded1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:55.353188 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f0100-c7ea-4336-a005-6232f4307dd6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:55.433808 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:55.472894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:55.496622 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 299/595] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:14:55.526570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f0100-c7ea-4336-a005-6232f4307dd6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:55.571813 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1e71c-9f62-4e03-b94f-2c5cb2e78e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:55.572844 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1e71c-9f62-4e03-b94f-2c5cb2e78e77 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.573343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1e71c-9f62-4e03-b94f-2c5cb2e78e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.573827 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1e71c-9f62-4e03-b94f-2c5cb2e78e77 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.574739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1e71c-9f62-4e03-b94f-2c5cb2e78e77 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:55.575255 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1e71c-9f62-4e03-b94f-2c5cb2e78e77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:55.577004 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1e71c-9f62-4e03-b94f-2c5cb2e78e77 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:55.586025 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 297/596] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Nov 2 07:14:55 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:55.815997 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01132b97-d66b-4bd6-88f0-bef637a4ded1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:55.881339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1e41a-e50a-45b8-9919-41819c35c8c6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.881670 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1e41a-e50a-45b8-9919-41819c35c8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.881891 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1e41a-e50a-45b8-9919-41819c35c8c6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.883341 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 298/597] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:55.888896 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbd6bc-5230-478b-a8f8-b3819cdea0f4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.889183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbd6bc-5230-478b-a8f8-b3819cdea0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.889447 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbd6bc-5230-478b-a8f8-b3819cdea0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.922524 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-51cbd6bc-5230-478b-a8f8-b3819cdea0f4 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:55.923794 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-51cbd6bc-5230-478b-a8f8-b3819cdea0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:55.967037 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01132b97-d66b-4bd6-88f0-bef637a4ded1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:56.006035 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a71c2-d543-4bf4-9c9b-b228436a713a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:56.006948 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a71c2-d543-4bf4-9c9b-b228436a713a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.007376 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a71c2-d543-4bf4-9c9b-b228436a713a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.007796 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a71c2-d543-4bf4-9c9b-b228436a713a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.017774 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 299/598] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:14:56.037160 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a71c2-d543-4bf4-9c9b-b228436a713a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7a4a71c2-d543-4bf4-9c9b-b228436a713a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:56.042214 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:56.043313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:14:56.044347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:56.046721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 300/599] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:14:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:56.052735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbd6bc-5230-478b-a8f8-b3819cdea0f4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:56.160987 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:56.243756 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbd6bc-5230-478b-a8f8-b3819cdea0f4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:56.285830 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bc1dc-aa5a-453a-bdac-2f9f8ec97a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:56.286438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bc1dc-aa5a-453a-bdac-2f9f8ec97a1e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.286714 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bc1dc-aa5a-453a-bdac-2f9f8ec97a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.286982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bc1dc-aa5a-453a-bdac-2f9f8ec97a1e None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.287641 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bc1dc-aa5a-453a-bdac-2f9f8ec97a1e None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:56.287939 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bc1dc-aa5a-453a-bdac-2f9f8ec97a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:56.289389 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bc1dc-aa5a-453a-bdac-2f9f8ec97a1e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:56.298499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 300/600] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:14:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:56.304856 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a71c2-d543-4bf4-9c9b-b228436a713a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:56.331530 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bc1dc-aa5a-453a-bdac-2f9f8ec97a1e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:56.361690 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:56.454770 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a71c2-d543-4bf4-9c9b-b228436a713a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:56.459906 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bc1dc-aa5a-453a-bdac-2f9f8ec97a1e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:56.474362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:56.495200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464aa27-3dd1-4a2c-89de-8f3ce26bcec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:56.495912 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464aa27-3dd1-4a2c-89de-8f3ce26bcec3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.496197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464aa27-3dd1-4a2c-89de-8f3ce26bcec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.496468 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464aa27-3dd1-4a2c-89de-8f3ce26bcec3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.508959 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520de51-1f0c-42bc-bb4a-ab6455f74295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:56.509665 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520de51-1f0c-42bc-bb4a-ab6455f74295 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.510012 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520de51-1f0c-42bc-bb4a-ab6455f74295 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.510421 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520de51-1f0c-42bc-bb4a-ab6455f74295 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.511421 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464aa27-3dd1-4a2c-89de-8f3ce26bcec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:56.513370 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464aa27-3dd1-4a2c-89de-8f3ce26bcec3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:56.522491 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520de51-1f0c-42bc-bb4a-ab6455f74295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6520de51-1f0c-42bc-bb4a-ab6455f74295 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:56.527988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:56.529152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:56.530215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:56.536774 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 301/601] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:56.536774 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d090186-431d-4248-bc45-94e0ff77584b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.536774 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d090186-431d-4248-bc45-94e0ff77584b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.536774 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d090186-431d-4248-bc45-94e0ff77584b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a464aa27-3dd1-4a2c-89de-8f3ce26bcec3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:56.539242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:56.540304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:56.541478 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:56.541478 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:56.541478 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:56.541685 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 302/602] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:56.542370 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d090186-431d-4248-bc45-94e0ff77584b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:56.543508 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 301/603] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:56.549666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff4ed0-24f7-447f-9ff8-0b0a471033a2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.549871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff4ed0-24f7-447f-9ff8-0b0a471033a2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.550094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff4ed0-24f7-447f-9ff8-0b0a471033a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.569092 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6cff4ed0-24f7-447f-9ff8-0b0a471033a2 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:56.571176 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6cff4ed0-24f7-447f-9ff8-0b0a471033a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:56.648207 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 302/604] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:14:56.653402 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff4ed0-24f7-447f-9ff8-0b0a471033a2 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:56.680065 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d090186-431d-4248-bc45-94e0ff77584b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:56.690484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:56.752468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75f73d-58cc-4f9c-81be-7522e969e489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:56.754254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75f73d-58cc-4f9c-81be-7522e969e489 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.755489 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75f73d-58cc-4f9c-81be-7522e969e489 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.755983 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75f73d-58cc-4f9c-81be-7522e969e489 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.757037 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75f73d-58cc-4f9c-81be-7522e969e489 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:56.758788 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75f73d-58cc-4f9c-81be-7522e969e489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:56.760371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75f73d-58cc-4f9c-81be-7522e969e489 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:56.778042 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 303/605] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:56.787716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75f73d-58cc-4f9c-81be-7522e969e489 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:56.850637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:56.872219 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff4ed0-24f7-447f-9ff8-0b0a471033a2 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:56.939093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936cdec-dd66-45e3-b52d-c30788b622ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:56.940642 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936cdec-dd66-45e3-b52d-c30788b622ad None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.941140 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936cdec-dd66-45e3-b52d-c30788b622ad None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.941602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936cdec-dd66-45e3-b52d-c30788b622ad None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.942453 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936cdec-dd66-45e3-b52d-c30788b622ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:56.942934 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936cdec-dd66-45e3-b52d-c30788b622ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:56.944486 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936cdec-dd66-45e3-b52d-c30788b622ad None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:56.956710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 303/606] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:56.960438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75f73d-58cc-4f9c-81be-7522e969e489 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:56.962426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936cdec-dd66-45e3-b52d-c30788b622ad None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.006288 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7eb03-9cb6-4e4d-8926-4e6ead16f894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.006842 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7eb03-9cb6-4e4d-8926-4e6ead16f894 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.007122 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7eb03-9cb6-4e4d-8926-4e6ead16f894 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.007395 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7eb03-9cb6-4e4d-8926-4e6ead16f894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.018849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7eb03-9cb6-4e4d-8926-4e6ead16f894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.020405 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7eb03-9cb6-4e4d-8926-4e6ead16f894 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3df7eb03-9cb6-4e4d-8926-4e6ead16f894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:57.035993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:57.037198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:57.038104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:57.038104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:57.038104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:57.038104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 304/607] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:57.038460 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7eb03-9cb6-4e4d-8926-4e6ead16f894 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.050857 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.086202 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936cdec-dd66-45e3-b52d-c30788b622ad None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.131428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.137732 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dfc75-ae02-4099-b8a8-42c9e7ce7e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.138395 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dfc75-ae02-4099-b8a8-42c9e7ce7e9b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.138674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dfc75-ae02-4099-b8a8-42c9e7ce7e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.138947 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dfc75-ae02-4099-b8a8-42c9e7ce7e9b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.152889 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dfc75-ae02-4099-b8a8-42c9e7ce7e9b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.154303 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7eb03-9cb6-4e4d-8926-4e6ead16f894 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.155034 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dfc75-ae02-4099-b8a8-42c9e7ce7e9b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-275dfc75-ae02-4099-b8a8-42c9e7ce7e9b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:57.172729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:57.174317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:57.175269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:57.175269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:57.175269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:57.175269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:57.175269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:57.175269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:57.175269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:57.175269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:57.175269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:14:57.175269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:57.183311 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 304/608] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Nov 2 07:14:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:57.183311 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dfc75-ae02-4099-b8a8-42c9e7ce7e9b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.197808 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8447d32-332b-4f24-9bd4-d8b27ee006eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.198441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447d32-332b-4f24-9bd4-d8b27ee006eb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.198719 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447d32-332b-4f24-9bd4-d8b27ee006eb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.198990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447d32-332b-4f24-9bd4-d8b27ee006eb None admin] PATH_INFO: `/v3/services/network` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.199570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8447d32-332b-4f24-9bd4-d8b27ee006eb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.201052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8447d32-332b-4f24-9bd4-d8b27ee006eb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f8447d32-332b-4f24-9bd4-d8b27ee006eb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:57.210880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:57.212152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:14:57.213030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:57.216730 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 305/609] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:57.217827 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8447d32-332b-4f24-9bd4-d8b27ee006eb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.311286 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dfc75-ae02-4099-b8a8-42c9e7ce7e9b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.327445 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8447d32-332b-4f24-9bd4-d8b27ee006eb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.347798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded25ce-7212-46b2-b715-6c197b02311b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.348415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded25ce-7212-46b2-b715-6c197b02311b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.348731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded25ce-7212-46b2-b715-6c197b02311b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.349009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded25ce-7212-46b2-b715-6c197b02311b None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.349648 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded25ce-7212-46b2-b715-6c197b02311b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:57.349952 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded25ce-7212-46b2-b715-6c197b02311b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.351306 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded25ce-7212-46b2-b715-6c197b02311b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.353160 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.363525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb20594-c039-45f4-ba01-d3262a450677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.364168 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb20594-c039-45f4-ba01-d3262a450677 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.364449 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb20594-c039-45f4-ba01-d3262a450677 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.364751 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb20594-c039-45f4-ba01-d3262a450677 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.370061 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 305/610] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:57.370188 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb20594-c039-45f4-ba01-d3262a450677 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:57.370188 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb20594-c039-45f4-ba01-d3262a450677 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.370341 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded25ce-7212-46b2-b715-6c197b02311b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.370423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb20594-c039-45f4-ba01-d3262a450677 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.386199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 306/611] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:57.387485 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f8ed2-cb0a-4d27-a098-8cb3e0604871 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.387767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f8ed2-cb0a-4d27-a098-8cb3e0604871 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.388040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f8ed2-cb0a-4d27-a098-8cb3e0604871 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.392789 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 307/612] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:57.392789 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f8ed2-cb0a-4d27-a098-8cb3e0604871 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.400317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.478354 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded25ce-7212-46b2-b715-6c197b02311b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.497369 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f8ed2-cb0a-4d27-a098-8cb3e0604871 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.514073 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e79ce2-98fd-47b1-aefd-ee646d67e20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.514694 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e79ce2-98fd-47b1-aefd-ee646d67e20c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.514977 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e79ce2-98fd-47b1-aefd-ee646d67e20c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.515258 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e79ce2-98fd-47b1-aefd-ee646d67e20c None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.515876 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e79ce2-98fd-47b1-aefd-ee646d67e20c None admin] RBAC: Adding query filter params (name=network) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:57.516183 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e79ce2-98fd-47b1-aefd-ee646d67e20c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.517722 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e79ce2-98fd-47b1-aefd-ee646d67e20c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.526809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 306/613] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:57.526809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e79ce2-98fd-47b1-aefd-ee646d67e20c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.538909 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e812d3-931f-4aea-85fc-fb6dd4ef68df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.539623 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e812d3-931f-4aea-85fc-fb6dd4ef68df None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.539905 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e812d3-931f-4aea-85fc-fb6dd4ef68df None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.540176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e812d3-931f-4aea-85fc-fb6dd4ef68df None admin] PATH_INFO: `/v3/users/swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.547599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e812d3-931f-4aea-85fc-fb6dd4ef68df None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.549129 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e812d3-931f-4aea-85fc-fb6dd4ef68df None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-26e812d3-931f-4aea-85fc-fb6dd4ef68df None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:14:57.561276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:57.562454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:57.563647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:57.563647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:57.563647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:14:57.563647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:14:57.563647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:14:57.563647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:14:57.563647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:14:57.563647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:14:57.563647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:14:57.563647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:57.567616 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 308/614] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:57.568884 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dfade-fbfc-465f-bc8a-f48563a3941d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.569168 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dfade-fbfc-465f-bc8a-f48563a3941d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.569443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dfade-fbfc-465f-bc8a-f48563a3941d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.587438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-400dfade-fbfc-465f-bc8a-f48563a3941d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:57.588650 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-400dfade-fbfc-465f-bc8a-f48563a3941d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:57.592392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.638587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e79ce2-98fd-47b1-aefd-ee646d67e20c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.656354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.664578 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 309/615] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:14:57.677430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dfade-fbfc-465f-bc8a-f48563a3941d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.678174 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10ecb9-e511-446f-8908-257fdbebf27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.678797 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10ecb9-e511-446f-8908-257fdbebf27a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.679077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10ecb9-e511-446f-8908-257fdbebf27a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.679350 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10ecb9-e511-446f-8908-257fdbebf27a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.751399 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10ecb9-e511-446f-8908-257fdbebf27a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.753306 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10ecb9-e511-446f-8908-257fdbebf27a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.812973 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.824071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dfade-fbfc-465f-bc8a-f48563a3941d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.864236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 307/616] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:14:57.864967 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10ecb9-e511-446f-8908-257fdbebf27a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.870374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3983f9-e4f7-40b3-a2b2-dd59b86024aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.870999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3983f9-e4f7-40b3-a2b2-dd59b86024aa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.871328 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3983f9-e4f7-40b3-a2b2-dd59b86024aa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.871606 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3983f9-e4f7-40b3-a2b2-dd59b86024aa None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.872238 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3983f9-e4f7-40b3-a2b2-dd59b86024aa None admin] RBAC: Adding query filter params (type=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:57.872565 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3983f9-e4f7-40b3-a2b2-dd59b86024aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.873944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3983f9-e4f7-40b3-a2b2-dd59b86024aa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.884327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 310/617] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:57.884327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3983f9-e4f7-40b3-a2b2-dd59b86024aa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.959491 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.978019 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10ecb9-e511-446f-8908-257fdbebf27a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.004359 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3983f9-e4f7-40b3-a2b2-dd59b86024aa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.019988 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2b83f-72a3-4bd7-b2dc-3dd048b43523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.020822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2b83f-72a3-4bd7-b2dc-3dd048b43523 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.020950 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2b83f-72a3-4bd7-b2dc-3dd048b43523 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.021176 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2b83f-72a3-4bd7-b2dc-3dd048b43523 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.021997 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2b83f-72a3-4bd7-b2dc-3dd048b43523 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:58.022284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2b83f-72a3-4bd7-b2dc-3dd048b43523 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.023627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2b83f-72a3-4bd7-b2dc-3dd048b43523 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.041526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 308/618] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:58.042391 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2b83f-72a3-4bd7-b2dc-3dd048b43523 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.048336 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532fb78-3acd-41ea-b065-613774636839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.050354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532fb78-3acd-41ea-b065-613774636839 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.050643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532fb78-3acd-41ea-b065-613774636839 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.050990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532fb78-3acd-41ea-b065-613774636839 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.061950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532fb78-3acd-41ea-b065-613774636839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.063569 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532fb78-3acd-41ea-b065-613774636839 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.066383 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-0532fb78-3acd-41ea-b065-613774636839 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:58.080885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:58.081895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:58.082850 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:58.082850 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:58.082850 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:58.082850 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 311/619] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:58.082850 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532fb78-3acd-41ea-b065-613774636839 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.157054 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2b83f-72a3-4bd7-b2dc-3dd048b43523 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.194900 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532fb78-3acd-41ea-b065-613774636839 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.197792 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d507-075d-48af-8baf-a9952342e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.198407 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d507-075d-48af-8baf-a9952342e8f6 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.198682 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d507-075d-48af-8baf-a9952342e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.198951 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d507-075d-48af-8baf-a9952342e8f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.199604 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30d507-075d-48af-8baf-a9952342e8f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.200993 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30d507-075d-48af-8baf-a9952342e8f6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.209357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.228487 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 309/620] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Nov 2 07:14:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:58.229597 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d507-075d-48af-8baf-a9952342e8f6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.264656 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e415f-3984-48c7-a3fc-c952dc72d760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.265691 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e415f-3984-48c7-a3fc-c952dc72d760 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.266427 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e415f-3984-48c7-a3fc-c952dc72d760 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.266427 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e415f-3984-48c7-a3fc-c952dc72d760 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.266831 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e415f-3984-48c7-a3fc-c952dc72d760 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.268302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e415f-3984-48c7-a3fc-c952dc72d760 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-fb5e415f-3984-48c7-a3fc-c952dc72d760 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:58.278573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:58.279668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:14:58.280525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:58.285618 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 312/621] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:58.286619 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e415f-3984-48c7-a3fc-c952dc72d760 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.328448 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.369860 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d507-075d-48af-8baf-a9952342e8f6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.425702 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e415f-3984-48c7-a3fc-c952dc72d760 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.426229 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b250e2-d3a0-431c-b057-0e56955523a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.428089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b250e2-d3a0-431c-b057-0e56955523a0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.428089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b250e2-d3a0-431c-b057-0e56955523a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.428089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b250e2-d3a0-431c-b057-0e56955523a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.475155 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205eb5cf-8678-405d-8ffa-d6d073153cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.475778 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205eb5cf-8678-405d-8ffa-d6d073153cca None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.476059 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205eb5cf-8678-405d-8ffa-d6d073153cca None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.476333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205eb5cf-8678-405d-8ffa-d6d073153cca None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.477076 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205eb5cf-8678-405d-8ffa-d6d073153cca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:58.477550 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205eb5cf-8678-405d-8ffa-d6d073153cca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.478757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205eb5cf-8678-405d-8ffa-d6d073153cca None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.491868 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 313/622] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:58.493512 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0fd7d-8852-4840-a79a-3de8917ff3e6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.493761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0fd7d-8852-4840-a79a-3de8917ff3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.494039 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0fd7d-8852-4840-a79a-3de8917ff3e6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.496597 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 314/623] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:58.496597 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0fd7d-8852-4840-a79a-3de8917ff3e6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.528058 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b250e2-d3a0-431c-b057-0e56955523a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.529713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b250e2-d3a0-431c-b057-0e56955523a0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.532476 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.600971 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.608246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0fd7d-8852-4840-a79a-3de8917ff3e6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.632373 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 310/624] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:14:58.633254 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b250e2-d3a0-431c-b057-0e56955523a0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.643513 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3614c77-1716-45c1-ae9f-f8f1c3d77a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.644127 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3614c77-1716-45c1-ae9f-f8f1c3d77a49 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.644403 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3614c77-1716-45c1-ae9f-f8f1c3d77a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.644704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3614c77-1716-45c1-ae9f-f8f1c3d77a49 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.645332 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3614c77-1716-45c1-ae9f-f8f1c3d77a49 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:58.645662 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3614c77-1716-45c1-ae9f-f8f1c3d77a49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.647012 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3614c77-1716-45c1-ae9f-f8f1c3d77a49 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.651595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 315/625] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:58.652749 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68749d-7782-4833-9435-fdc3ee87f99d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.653028 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68749d-7782-4833-9435-fdc3ee87f99d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.653298 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68749d-7782-4833-9435-fdc3ee87f99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.670531 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4e68749d-7782-4833-9435-fdc3ee87f99d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:58.671710 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4e68749d-7782-4833-9435-fdc3ee87f99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:58.737770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 316/626] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:14:58.738574 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b250e2-d3a0-431c-b057-0e56955523a0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.738694 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68749d-7782-4833-9435-fdc3ee87f99d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.765573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.776492 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c644b-9405-4f4a-89cf-71e642473ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.777150 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c644b-9405-4f4a-89cf-71e642473ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.777432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c644b-9405-4f4a-89cf-71e642473ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.777703 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c644b-9405-4f4a-89cf-71e642473ba4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.785574 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c644b-9405-4f4a-89cf-71e642473ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.787281 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c644b-9405-4f4a-89cf-71e642473ba4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-113c644b-9405-4f4a-89cf-71e642473ba4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:58.799184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:58.800077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:58.801009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:58.801009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:58.801009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:58.801009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:58.801009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:58.801009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:58.801009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:58.801009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:58.801009 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 311/627] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:58.801503 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c644b-9405-4f4a-89cf-71e642473ba4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.836196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.850381 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68749d-7782-4833-9435-fdc3ee87f99d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.888542 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2069b-64aa-4207-b320-79e665662def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.889282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2069b-64aa-4207-b320-79e665662def None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.889675 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2069b-64aa-4207-b320-79e665662def None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.889949 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2069b-64aa-4207-b320-79e665662def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.901068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2069b-64aa-4207-b320-79e665662def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.902707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2069b-64aa-4207-b320-79e665662def None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.906748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c644b-9405-4f4a-89cf-71e642473ba4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bbf2069b-64aa-4207-b320-79e665662def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:58.918577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:58.919543 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:58.920435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:58.920435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:58.920435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:14:58.920435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 317/628] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:58.920704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2069b-64aa-4207-b320-79e665662def None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.945288 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97a46-30f7-4152-9fc7-6b77a5a21b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.946142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97a46-30f7-4152-9fc7-6b77a5a21b77 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.946425 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97a46-30f7-4152-9fc7-6b77a5a21b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.946810 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97a46-30f7-4152-9fc7-6b77a5a21b77 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.947320 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea97a46-30f7-4152-9fc7-6b77a5a21b77 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:58.947627 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea97a46-30f7-4152-9fc7-6b77a5a21b77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.949049 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea97a46-30f7-4152-9fc7-6b77a5a21b77 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.958406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 312/629] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:58.958406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97a46-30f7-4152-9fc7-6b77a5a21b77 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.008163 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.030624 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2069b-64aa-4207-b320-79e665662def None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.051426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.057676 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97a46-30f7-4152-9fc7-6b77a5a21b77 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.064762 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1fa57-e2c7-44bc-a3d7-208eda987377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.065364 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1fa57-e2c7-44bc-a3d7-208eda987377 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.065637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1fa57-e2c7-44bc-a3d7-208eda987377 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.065905 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1fa57-e2c7-44bc-a3d7-208eda987377 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.066582 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1fa57-e2c7-44bc-a3d7-208eda987377 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:59.066885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1fa57-e2c7-44bc-a3d7-208eda987377 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.068310 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1fa57-e2c7-44bc-a3d7-208eda987377 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.079911 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 318/630] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:59.080718 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1fa57-e2c7-44bc-a3d7-208eda987377 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.091654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a69088-e2f4-4464-864a-8c9543bf0cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.092524 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a69088-e2f4-4464-864a-8c9543bf0cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.092524 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a69088-e2f4-4464-864a-8c9543bf0cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.092837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a69088-e2f4-4464-864a-8c9543bf0cc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.093566 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a69088-e2f4-4464-864a-8c9543bf0cc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:59.093864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a69088-e2f4-4464-864a-8c9543bf0cc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.095213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a69088-e2f4-4464-864a-8c9543bf0cc9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.106917 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 313/631] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:59.107678 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a69088-e2f4-4464-864a-8c9543bf0cc9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.177737 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1fa57-e2c7-44bc-a3d7-208eda987377 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.203826 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a69088-e2f4-4464-864a-8c9543bf0cc9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.210728 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875c083-1452-4ce2-93ca-6e84d4def328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.211335 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875c083-1452-4ce2-93ca-6e84d4def328 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.211606 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875c083-1452-4ce2-93ca-6e84d4def328 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.212127 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875c083-1452-4ce2-93ca-6e84d4def328 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.212816 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875c083-1452-4ce2-93ca-6e84d4def328 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.214141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875c083-1452-4ce2-93ca-6e84d4def328 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.223267 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.235491 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 319/632] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Nov 2 07:14:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:59.236984 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875c083-1452-4ce2-93ca-6e84d4def328 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.242899 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84f42a-0c75-440b-a3c6-bcbc608187bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.243531 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84f42a-0c75-440b-a3c6-bcbc608187bc None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.243808 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84f42a-0c75-440b-a3c6-bcbc608187bc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.244076 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84f42a-0c75-440b-a3c6-bcbc608187bc None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.244765 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84f42a-0c75-440b-a3c6-bcbc608187bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.246206 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84f42a-0c75-440b-a3c6-bcbc608187bc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.261820 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 314/633] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:14:59 2022] POST /identity/v3/services => generated 269 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:59.262715 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84f42a-0c75-440b-a3c6-bcbc608187bc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.284645 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.384898 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875c083-1452-4ce2-93ca-6e84d4def328 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.409000 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84f42a-0c75-440b-a3c6-bcbc608187bc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.433113 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50df6fa-fe07-4f2d-8f10-a484244cb5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.433761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50df6fa-fe07-4f2d-8f10-a484244cb5ee None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.433996 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50df6fa-fe07-4f2d-8f10-a484244cb5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.434272 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50df6fa-fe07-4f2d-8f10-a484244cb5ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.435020 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50df6fa-fe07-4f2d-8f10-a484244cb5ee None admin] RBAC: Adding query filter params (role.id=d276003aacd44f11a8b5e733a9a54f02, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=b67332ed32044f94a1b3a6a6e6bbb348) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:59.435828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50df6fa-fe07-4f2d-8f10-a484244cb5ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.439313 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50df6fa-fe07-4f2d-8f10-a484244cb5ee None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.445500 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 320/634] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/role_assignments?user.id=b67332ed32044f94a1b3a6a6e6bbb348&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=d276003aacd44f11a8b5e733a9a54f02 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:59.464423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0c3a6-1480-4cfa-aaed-b2193e70df9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.465240 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0c3a6-1480-4cfa-aaed-b2193e70df9f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.465523 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0c3a6-1480-4cfa-aaed-b2193e70df9f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.465795 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0c3a6-1480-4cfa-aaed-b2193e70df9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.474959 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0c3a6-1480-4cfa-aaed-b2193e70df9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.476822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0c3a6-1480-4cfa-aaed-b2193e70df9f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-41a0c3a6-1480-4cfa-aaed-b2193e70df9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:14:59.491440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:59.492526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:59.493936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:14:59.493936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:14:59.493936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:14:59.493936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:14:59.493936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:14:59.493936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:14:59.493936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:14:59.493936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:14:59.493936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 315/635] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:59.498858 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50df6fa-fe07-4f2d-8f10-a484244cb5ee None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.565336 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.612195 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50df6fa-fe07-4f2d-8f10-a484244cb5ee None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.656382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cf78a-eb23-4192-abcb-e425dc1d2639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.657028 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cf78a-eb23-4192-abcb-e425dc1d2639 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.657303 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cf78a-eb23-4192-abcb-e425dc1d2639 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.657574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cf78a-eb23-4192-abcb-e425dc1d2639 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.658820 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cf78a-eb23-4192-abcb-e425dc1d2639 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:59.659119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cf78a-eb23-4192-abcb-e425dc1d2639 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.660574 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cf78a-eb23-4192-abcb-e425dc1d2639 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.673670 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 321/636] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:59.679251 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0c3a6-1480-4cfa-aaed-b2193e70df9f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.749065 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.785879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0c3a6-1480-4cfa-aaed-b2193e70df9f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.822204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac733906-53fb-4d0f-9412-39e68f023420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.822818 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac733906-53fb-4d0f-9412-39e68f023420 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.823094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac733906-53fb-4d0f-9412-39e68f023420 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.823359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac733906-53fb-4d0f-9412-39e68f023420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.824074 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac733906-53fb-4d0f-9412-39e68f023420 None admin] RBAC: Adding query filter params (role.id=81d287a8b606458d87c9607e8b48ac58, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=7acc23ec5ccf4756a9e00403116ffe6f) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:59.824375 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac733906-53fb-4d0f-9412-39e68f023420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.825888 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac733906-53fb-4d0f-9412-39e68f023420 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.832159 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 316/637] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/role_assignments?user.id=7acc23ec5ccf4756a9e00403116ffe6f&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=81d287a8b606458d87c9607e8b48ac58 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:01.495332 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee41dd3-4f87-4e23-8d07-9878a3bad2a6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.495851 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee41dd3-4f87-4e23-8d07-9878a3bad2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.495851 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee41dd3-4f87-4e23-8d07-9878a3bad2a6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.497645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 322/638] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:01.502980 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4f860-b646-40e1-88d5-331ff058c09c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.503378 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4f860-b646-40e1-88d5-331ff058c09c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.503518 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4f860-b646-40e1-88d5-331ff058c09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.522533 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f7b4f860-b646-40e1-88d5-331ff058c09c None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:01.523740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f7b4f860-b646-40e1-88d5-331ff058c09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:01.563383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc5909-ae80-4ab6-b2ac-f343e72c770e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.563699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc5909-ae80-4ab6-b2ac-f343e72c770e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.563934 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc5909-ae80-4ab6-b2ac-f343e72c770e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.565695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 323/639] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:01.571191 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc5fd-7f1b-4898-bf01-14b062d31071 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.571472 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc5fd-7f1b-4898-bf01-14b062d31071 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.571742 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc5fd-7f1b-4898-bf01-14b062d31071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.598036 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3cfbc5fd-7f1b-4898-bf01-14b062d31071 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:01.599629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3cfbc5fd-7f1b-4898-bf01-14b062d31071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:01.600747 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 317/640] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:01.682096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 324/641] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:01.712713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069e452-6377-49c6-a7db-b12695a03939 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.713007 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069e452-6377-49c6-a7db-b12695a03939 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.713302 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069e452-6377-49c6-a7db-b12695a03939 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.714905 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 318/642] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:01.720196 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c4ba2-7be9-469f-833f-d6c2efa6579e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.720749 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c4ba2-7be9-469f-833f-d6c2efa6579e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.721164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c4ba2-7be9-469f-833f-d6c2efa6579e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.740255 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-572c4ba2-7be9-469f-833f-d6c2efa6579e None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:01.741645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-572c4ba2-7be9-469f-833f-d6c2efa6579e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:01.786181 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:01.811331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 325/643] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:01.870659 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069e452-6377-49c6-a7db-b12695a03939 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:01.939720 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:01.960441 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c4ba2-7be9-469f-833f-d6c2efa6579e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:01.983807 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069e452-6377-49c6-a7db-b12695a03939 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.023171 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af8eb1-389a-44c1-be4a-f771a430f45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.023823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8eb1-389a-44c1-be4a-f771a430f45d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.024313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8eb1-389a-44c1-be4a-f771a430f45d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.024607 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8eb1-389a-44c1-be4a-f771a430f45d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.025586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af8eb1-389a-44c1-be4a-f771a430f45d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.027451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af8eb1-389a-44c1-be4a-f771a430f45d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-84af8eb1-389a-44c1-be4a-f771a430f45d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:02.039232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:02.040248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:02.041236 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 319/644] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:02.042025 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d4ae2-e9d9-4dfe-8432-f78cfe95eb04 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.042025 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d4ae2-e9d9-4dfe-8432-f78cfe95eb04 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.042664 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d4ae2-e9d9-4dfe-8432-f78cfe95eb04 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.043541 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 320/645] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:02.049815 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d4ae2-e9d9-4dfe-8432-f78cfe95eb04 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.072902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c4ba2-7be9-469f-833f-d6c2efa6579e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.109379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1157712-b607-4687-9518-1e78fd6cce59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.110009 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1157712-b607-4687-9518-1e78fd6cce59 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.110286 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1157712-b607-4687-9518-1e78fd6cce59 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.110551 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1157712-b607-4687-9518-1e78fd6cce59 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.111250 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1157712-b607-4687-9518-1e78fd6cce59 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.112629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1157712-b607-4687-9518-1e78fd6cce59 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d1157712-b607-4687-9518-1e78fd6cce59 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:02.121879 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:02.122902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:15:02.123824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:02.124981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 326/646] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Nov 2 07:15:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:02.125940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ffe7c-04b0-4eae-839b-92b43a7c660e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.126340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ffe7c-04b0-4eae-839b-92b43a7c660e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.127116 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ffe7c-04b0-4eae-839b-92b43a7c660e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.144729 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d17ffe7c-04b0-4eae-839b-92b43a7c660e None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:02.145909 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d17ffe7c-04b0-4eae-839b-92b43a7c660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:02.152219 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d4ae2-e9d9-4dfe-8432-f78cfe95eb04 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.176654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.179471 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.186073 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e06c2b-bc62-4c53-b39c-4d7c89565f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.186680 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e06c2b-bc62-4c53-b39c-4d7c89565f0d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.186956 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e06c2b-bc62-4c53-b39c-4d7c89565f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.187223 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e06c2b-bc62-4c53-b39c-4d7c89565f0d None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.187834 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e06c2b-bc62-4c53-b39c-4d7c89565f0d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:02.188131 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e06c2b-bc62-4c53-b39c-4d7c89565f0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.190117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e06c2b-bc62-4c53-b39c-4d7c89565f0d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.194675 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 321/647] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:02.195438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e06c2b-bc62-4c53-b39c-4d7c89565f0d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.211215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 327/648] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:02 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:02.212305 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ffe7c-04b0-4eae-839b-92b43a7c660e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.315511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e06c2b-bc62-4c53-b39c-4d7c89565f0d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.315692 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ffe7c-04b0-4eae-839b-92b43a7c660e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.350826 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9f3d-35d8-4716-93db-c6eaa567e049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.351003 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1ae62-df43-41ae-809d-2f0e1b9557e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.351517 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9f3d-35d8-4716-93db-c6eaa567e049 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.351622 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1ae62-df43-41ae-809d-2f0e1b9557e1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.351797 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9f3d-35d8-4716-93db-c6eaa567e049 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.351904 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1ae62-df43-41ae-809d-2f0e1b9557e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.352068 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9f3d-35d8-4716-93db-c6eaa567e049 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.352174 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1ae62-df43-41ae-809d-2f0e1b9557e1 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.352734 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1ae62-df43-41ae-809d-2f0e1b9557e1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:02.353032 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1ae62-df43-41ae-809d-2f0e1b9557e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.354378 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1ae62-df43-41ae-809d-2f0e1b9557e1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.363499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 328/649] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:02.363499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1ae62-df43-41ae-809d-2f0e1b9557e1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.367514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b9f3d-35d8-4716-93db-c6eaa567e049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d05b9f3d-35d8-4716-93db-c6eaa567e049 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:02.373352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:02.374252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:02.375093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 322/650] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:02.375784 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9f3d-35d8-4716-93db-c6eaa567e049 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.399997 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.428224 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.469676 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1ae62-df43-41ae-809d-2f0e1b9557e1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.482790 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9f3d-35d8-4716-93db-c6eaa567e049 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.504712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afa3a8-a7eb-4cc7-a6b0-ef6c30f855b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.505326 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afa3a8-a7eb-4cc7-a6b0-ef6c30f855b8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.505602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afa3a8-a7eb-4cc7-a6b0-ef6c30f855b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.505866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afa3a8-a7eb-4cc7-a6b0-ef6c30f855b8 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.506487 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afa3a8-a7eb-4cc7-a6b0-ef6c30f855b8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:02.506786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afa3a8-a7eb-4cc7-a6b0-ef6c30f855b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.508128 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afa3a8-a7eb-4cc7-a6b0-ef6c30f855b8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.512409 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 329/651] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:02.513202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afa3a8-a7eb-4cc7-a6b0-ef6c30f855b8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.516021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50520d6-3ff0-4dbb-b3ac-6314940cabd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.516682 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50520d6-3ff0-4dbb-b3ac-6314940cabd3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.516961 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50520d6-3ff0-4dbb-b3ac-6314940cabd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.517231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50520d6-3ff0-4dbb-b3ac-6314940cabd3 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.518458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50520d6-3ff0-4dbb-b3ac-6314940cabd3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:02.518761 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50520d6-3ff0-4dbb-b3ac-6314940cabd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.520216 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50520d6-3ff0-4dbb-b3ac-6314940cabd3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.525897 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 323/652] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:02.526767 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50520d6-3ff0-4dbb-b3ac-6314940cabd3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.638100 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.642062 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afa3a8-a7eb-4cc7-a6b0-ef6c30f855b8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.649530 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50520d6-3ff0-4dbb-b3ac-6314940cabd3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.666384 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.687707 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2c417-7f66-4a97-9910-8d2721c168e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.688344 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2c417-7f66-4a97-9910-8d2721c168e1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.688663 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2c417-7f66-4a97-9910-8d2721c168e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.688847 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bd0f5-494e-43d6-9b95-8321c3f724f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.688980 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2c417-7f66-4a97-9910-8d2721c168e1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.689961 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bd0f5-494e-43d6-9b95-8321c3f724f8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.690133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2c417-7f66-4a97-9910-8d2721c168e1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.690227 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bd0f5-494e-43d6-9b95-8321c3f724f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.690499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bd0f5-494e-43d6-9b95-8321c3f724f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.691474 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bd0f5-494e-43d6-9b95-8321c3f724f8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:02.691474 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bd0f5-494e-43d6-9b95-8321c3f724f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.691655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2c417-7f66-4a97-9910-8d2721c168e1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.693079 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bd0f5-494e-43d6-9b95-8321c3f724f8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ade2c417-7f66-4a97-9910-8d2721c168e1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:02.701085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:02.702079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:02.703069 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 324/653] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:02.703851 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2c417-7f66-4a97-9910-8d2721c168e1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.709118 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 330/654] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:02.710027 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bd0f5-494e-43d6-9b95-8321c3f724f8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.808801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2c417-7f66-4a97-9910-8d2721c168e1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.818880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bd0f5-494e-43d6-9b95-8321c3f724f8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.846351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35193eb6-045a-4cc3-994a-e9fa6040af42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.846952 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35193eb6-045a-4cc3-994a-e9fa6040af42 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.847225 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35193eb6-045a-4cc3-994a-e9fa6040af42 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.847585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35193eb6-045a-4cc3-994a-e9fa6040af42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.848186 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35193eb6-045a-4cc3-994a-e9fa6040af42 None admin] RBAC: Adding query filter params (interface=public, service_id=ae08d23c9a0f4f76b6c1f3db4e2509f0, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:02.848490 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35193eb6-045a-4cc3-994a-e9fa6040af42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.849865 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35193eb6-045a-4cc3-994a-e9fa6040af42 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.854843 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc073a8f-dac4-4b30-8698-33305b47e733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.855454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc073a8f-dac4-4b30-8698-33305b47e733 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.855730 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc073a8f-dac4-4b30-8698-33305b47e733 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.856079 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc073a8f-dac4-4b30-8698-33305b47e733 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.856746 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc073a8f-dac4-4b30-8698-33305b47e733 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:02.857038 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc073a8f-dac4-4b30-8698-33305b47e733 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.858387 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc073a8f-dac4-4b30-8698-33305b47e733 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.859660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 325/655] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/endpoints?service_id=ae08d23c9a0f4f76b6c1f3db4e2509f0&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:02.860503 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35193eb6-045a-4cc3-994a-e9fa6040af42 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.863472 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 331/656] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:02.864459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc073a8f-dac4-4b30-8698-33305b47e733 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.895822 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.909569 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.968630 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35193eb6-045a-4cc3-994a-e9fa6040af42 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.975934 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc073a8f-dac4-4b30-8698-33305b47e733 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.004577 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930051a-d1a4-48ff-94c0-0499eac7a874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.004577 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930051a-d1a4-48ff-94c0-0499eac7a874 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.004577 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930051a-d1a4-48ff-94c0-0499eac7a874 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.004903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930051a-d1a4-48ff-94c0-0499eac7a874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.014968 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f230dc-1618-43bb-825a-c67b58bb4405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.015434 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930051a-d1a4-48ff-94c0-0499eac7a874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.015603 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f230dc-1618-43bb-825a-c67b58bb4405 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.015844 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f230dc-1618-43bb-825a-c67b58bb4405 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.016110 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f230dc-1618-43bb-825a-c67b58bb4405 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.016729 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f230dc-1618-43bb-825a-c67b58bb4405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.016905 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930051a-d1a4-48ff-94c0-0499eac7a874 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.018616 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f230dc-1618-43bb-825a-c67b58bb4405 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.032566 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 332/657] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/services => generated 1029 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:15:03.032566 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f230dc-1618-43bb-825a-c67b58bb4405 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c930051a-d1a4-48ff-94c0-0499eac7a874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:03.032895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:03.034012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:03.035109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:03.035109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:03.035109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:03.038089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 326/658] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:03.038821 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930051a-d1a4-48ff-94c0-0499eac7a874 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.144875 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.155861 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f230dc-1618-43bb-825a-c67b58bb4405 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.161896 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.170569 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930051a-d1a4-48ff-94c0-0499eac7a874 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.194966 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772ee8e-fc19-4dcc-87ce-b2d0ff74dbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.195584 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772ee8e-fc19-4dcc-87ce-b2d0ff74dbae None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.195866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772ee8e-fc19-4dcc-87ce-b2d0ff74dbae None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.196135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772ee8e-fc19-4dcc-87ce-b2d0ff74dbae None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.196800 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772ee8e-fc19-4dcc-87ce-b2d0ff74dbae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:03.197167 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772ee8e-fc19-4dcc-87ce-b2d0ff74dbae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.198545 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772ee8e-fc19-4dcc-87ce-b2d0ff74dbae None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.208882 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 333/659] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:03.213474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27ce33-82aa-4fac-88af-13099e4eee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.214109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27ce33-82aa-4fac-88af-13099e4eee0f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.214384 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27ce33-82aa-4fac-88af-13099e4eee0f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.214652 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27ce33-82aa-4fac-88af-13099e4eee0f None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.215319 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27ce33-82aa-4fac-88af-13099e4eee0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:03.215634 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27ce33-82aa-4fac-88af-13099e4eee0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.217104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27ce33-82aa-4fac-88af-13099e4eee0f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.231494 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 327/660] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:03.237410 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772ee8e-fc19-4dcc-87ce-b2d0ff74dbae None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.352721 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772ee8e-fc19-4dcc-87ce-b2d0ff74dbae None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.389942 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d637f00-dfea-433b-b928-e2c19ddb22b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.391006 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d637f00-dfea-433b-b928-e2c19ddb22b7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.391495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d637f00-dfea-433b-b928-e2c19ddb22b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.391973 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d637f00-dfea-433b-b928-e2c19ddb22b7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.401084 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d637f00-dfea-433b-b928-e2c19ddb22b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.403005 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d637f00-dfea-433b-b928-e2c19ddb22b7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9d637f00-dfea-433b-b928-e2c19ddb22b7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:03.416957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:03.417936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:03.418863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:03.418863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:03.418863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:03.418863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:03.418863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:03.418863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:03.418863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:03.418863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:03.418863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:15:03.418863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:03.421674 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 334/661] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:03.427301 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27ce33-82aa-4fac-88af-13099e4eee0f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.536851 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27ce33-82aa-4fac-88af-13099e4eee0f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.576494 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae50231-a909-40aa-9b40-8f904ae7dc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.577497 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae50231-a909-40aa-9b40-8f904ae7dc4b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.577976 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae50231-a909-40aa-9b40-8f904ae7dc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.578462 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae50231-a909-40aa-9b40-8f904ae7dc4b None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.579387 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae50231-a909-40aa-9b40-8f904ae7dc4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:03.579973 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae50231-a909-40aa-9b40-8f904ae7dc4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.581617 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae50231-a909-40aa-9b40-8f904ae7dc4b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.601458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 328/662] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:03.607376 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d637f00-dfea-433b-b928-e2c19ddb22b7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.623896 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.723317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d637f00-dfea-433b-b928-e2c19ddb22b7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.757643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9efafc-26a4-4321-b3f9-3d149e3b98e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.758282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9efafc-26a4-4321-b3f9-3d149e3b98e5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.758560 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9efafc-26a4-4321-b3f9-3d149e3b98e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.758835 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9efafc-26a4-4321-b3f9-3d149e3b98e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.771379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9efafc-26a4-4321-b3f9-3d149e3b98e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.772934 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9efafc-26a4-4321-b3f9-3d149e3b98e5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ad9efafc-26a4-4321-b3f9-3d149e3b98e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:03.788633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:03.789614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:03.790501 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:03.790501 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:03.790501 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:03.792187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 335/663] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:03.797952 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae50231-a909-40aa-9b40-8f904ae7dc4b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.814035 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.901894 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae50231-a909-40aa-9b40-8f904ae7dc4b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.936577 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4a3bb-ef43-499d-b557-72565aef00ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.937411 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a3bb-ef43-499d-b557-72565aef00ff None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.937831 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a3bb-ef43-499d-b557-72565aef00ff None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.938243 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a3bb-ef43-499d-b557-72565aef00ff None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.939012 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4a3bb-ef43-499d-b557-72565aef00ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:03.939536 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4a3bb-ef43-499d-b557-72565aef00ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.941102 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4a3bb-ef43-499d-b557-72565aef00ff None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.952908 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 329/664] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:03.959008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9efafc-26a4-4321-b3f9-3d149e3b98e5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:04.026950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:04.064417 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9efafc-26a4-4321-b3f9-3d149e3b98e5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:04.099152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aec461-0979-407d-9851-c18b25a3ff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:04.099780 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aec461-0979-407d-9851-c18b25a3ff0c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.100056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aec461-0979-407d-9851-c18b25a3ff0c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.100323 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aec461-0979-407d-9851-c18b25a3ff0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.107079 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aec461-0979-407d-9851-c18b25a3ff0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:04.108502 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aec461-0979-407d-9851-c18b25a3ff0c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c8aec461-0979-407d-9851-c18b25a3ff0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:04.119773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:04.120732 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:04.121643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:04.121643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:04.121643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:04.121643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:04.121643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:04.121643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:04.121643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:04.121643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:04.121643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 336/665] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:04.127358 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4a3bb-ef43-499d-b557-72565aef00ff None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:04.207841 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:04.242993 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4a3bb-ef43-499d-b557-72565aef00ff None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:04.278919 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ab4bd-c66b-4360-978d-5c6e3a3d3e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:04.279848 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ab4bd-c66b-4360-978d-5c6e3a3d3e9f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.280334 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ab4bd-c66b-4360-978d-5c6e3a3d3e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.280847 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ab4bd-c66b-4360-978d-5c6e3a3d3e9f None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.281754 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ab4bd-c66b-4360-978d-5c6e3a3d3e9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:04.282274 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ab4bd-c66b-4360-978d-5c6e3a3d3e9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:04.284436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ab4bd-c66b-4360-978d-5c6e3a3d3e9f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:04.297768 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 330/666] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:04.304396 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aec461-0979-407d-9851-c18b25a3ff0c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:04.405690 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aec461-0979-407d-9851-c18b25a3ff0c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:04.432260 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:04.441498 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd7c30-cc83-4e49-803a-5bb80ce643d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:04.442591 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7c30-cc83-4e49-803a-5bb80ce643d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.443289 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7c30-cc83-4e49-803a-5bb80ce643d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.443971 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7c30-cc83-4e49-803a-5bb80ce643d3 None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/b67332ed32044f94a1b3a6a6e6bbb348/roles/d660bf006ef649159f7369c7f1623c04` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.446171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd7c30-cc83-4e49-803a-5bb80ce643d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=b67332ed32044f94a1b3a6a6e6bbb348, role_id=d660bf006ef649159f7369c7f1623c04)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:04.447587 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd7c30-cc83-4e49-803a-5bb80ce643d3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:04.463666 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 337/667] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:04 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/b67332ed32044f94a1b3a6a6e6bbb348/roles/d660bf006ef649159f7369c7f1623c04 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:04.701248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f5f89-40e2-46de-84a3-413031404963 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.701809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f5f89-40e2-46de-84a3-413031404963 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.701809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f5f89-40e2-46de-84a3-413031404963 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.703373 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 331/668] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:04.709617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db443b43-97bf-4b84-96aa-b60895e16e26 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.710035 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db443b43-97bf-4b84-96aa-b60895e16e26 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.710231 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db443b43-97bf-4b84-96aa-b60895e16e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.728510 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-db443b43-97bf-4b84-96aa-b60895e16e26 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:04.729726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-db443b43-97bf-4b84-96aa-b60895e16e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:04.799444 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 338/669] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:04 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:05.066613 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f5f89-40e2-46de-84a3-413031404963 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.176755 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f5f89-40e2-46de-84a3-413031404963 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.208287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.213742 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980694b9-f172-4bff-a732-88952c17ae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.214365 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980694b9-f172-4bff-a732-88952c17ae50 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.214649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980694b9-f172-4bff-a732-88952c17ae50 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.214922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980694b9-f172-4bff-a732-88952c17ae50 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.215652 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980694b9-f172-4bff-a732-88952c17ae50 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:05.217852 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980694b9-f172-4bff-a732-88952c17ae50 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:05.237495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 332/670] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:15:05 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:05.413515 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a7ca1-eb0f-41c1-88f7-42bb0178291b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.414070 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a7ca1-eb0f-41c1-88f7-42bb0178291b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.414484 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a7ca1-eb0f-41c1-88f7-42bb0178291b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.416351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 339/671] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:05.423004 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ac5ee-f06d-47b7-907b-e75bcd755685 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.423464 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ac5ee-f06d-47b7-907b-e75bcd755685 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.423844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ac5ee-f06d-47b7-907b-e75bcd755685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.442722 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0b1ac5ee-f06d-47b7-907b-e75bcd755685 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:05.443919 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0b1ac5ee-f06d-47b7-907b-e75bcd755685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:05.499882 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c1d4d-286d-424b-b56f-020497c33082 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.500196 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c1d4d-286d-424b-b56f-020497c33082 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.500436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c1d4d-286d-424b-b56f-020497c33082 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.502129 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 340/672] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:05.507338 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80db84f-3023-4c78-ae9d-f644e5ac2db6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.507979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80db84f-3023-4c78-ae9d-f644e5ac2db6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.508463 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80db84f-3023-4c78-ae9d-f644e5ac2db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.526557 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 333/673] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:05.527678 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f80db84f-3023-4c78-ae9d-f644e5ac2db6 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:05.529105 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f80db84f-3023-4c78-ae9d-f644e5ac2db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:05.601756 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 341/674] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:05.813092 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ac5ee-f06d-47b7-907b-e75bcd755685 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.877295 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80db84f-3023-4c78-ae9d-f644e5ac2db6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.918826 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ac5ee-f06d-47b7-907b-e75bcd755685 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.937099 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.938946 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.964620 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b33f2a-4c31-4ebd-99ad-03caa822956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.965265 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b33f2a-4c31-4ebd-99ad-03caa822956a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.965546 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b33f2a-4c31-4ebd-99ad-03caa822956a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.965819 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b33f2a-4c31-4ebd-99ad-03caa822956a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.966402 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b33f2a-4c31-4ebd-99ad-03caa822956a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:05.967791 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b33f2a-4c31-4ebd-99ad-03caa822956a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e2b33f2a-4c31-4ebd-99ad-03caa822956a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:05.977415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:05.978555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:05.979445 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 334/675] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:05.984266 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b33f2a-4c31-4ebd-99ad-03caa822956a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:06.015162 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80db84f-3023-4c78-ae9d-f644e5ac2db6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:06.054472 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa2293-5923-4c34-8ae7-4f7861a53e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:06.055083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa2293-5923-4c34-8ae7-4f7861a53e59 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.055368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa2293-5923-4c34-8ae7-4f7861a53e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.055639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa2293-5923-4c34-8ae7-4f7861a53e59 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.056299 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa2293-5923-4c34-8ae7-4f7861a53e59 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:06.057783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa2293-5923-4c34-8ae7-4f7861a53e59 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:06.076700 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 342/676] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:15:05 2022] POST /identity/v3/services => generated 241 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:06.123490 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b33f2a-4c31-4ebd-99ad-03caa822956a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:06.172419 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028fbca-bf2d-428b-90fc-41f37e6fa4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:06.174020 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028fbca-bf2d-428b-90fc-41f37e6fa4aa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.174558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028fbca-bf2d-428b-90fc-41f37e6fa4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.175131 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028fbca-bf2d-428b-90fc-41f37e6fa4aa None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.176021 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028fbca-bf2d-428b-90fc-41f37e6fa4aa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:06.176582 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028fbca-bf2d-428b-90fc-41f37e6fa4aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:06.178331 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028fbca-bf2d-428b-90fc-41f37e6fa4aa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:06.188757 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 335/677] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:06.190928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa2293-5923-4c34-8ae7-4f7861a53e59 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:06.309440 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa2293-5923-4c34-8ae7-4f7861a53e59 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:06.317653 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:06.349532 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd00e7-4470-4d45-99d8-853dc7d6d8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:06.350071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd00e7-4470-4d45-99d8-853dc7d6d8cc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.350347 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd00e7-4470-4d45-99d8-853dc7d6d8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.350617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd00e7-4470-4d45-99d8-853dc7d6d8cc None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.351244 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd00e7-4470-4d45-99d8-853dc7d6d8cc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:06.351548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd00e7-4470-4d45-99d8-853dc7d6d8cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:06.352939 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd00e7-4470-4d45-99d8-853dc7d6d8cc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:06.358218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 343/678] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Nov 2 07:15:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:06.364030 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028fbca-bf2d-428b-90fc-41f37e6fa4aa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:06.411432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:06.470798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028fbca-bf2d-428b-90fc-41f37e6fa4aa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:06.506169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3cdaa-5334-41f9-9658-3a4c761cd131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:06.506786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3cdaa-5334-41f9-9658-3a4c761cd131 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.507065 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3cdaa-5334-41f9-9658-3a4c761cd131 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.507339 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3cdaa-5334-41f9-9658-3a4c761cd131 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.508024 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3cdaa-5334-41f9-9658-3a4c761cd131 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:06.509489 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3cdaa-5334-41f9-9658-3a4c761cd131 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:06.530970 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 336/679] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Nov 2 07:15:06 2022] POST /identity/v3/endpoints => generated 347 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:06.760289 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639db17c-f9e1-4394-a327-a1eb272d034e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.760633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639db17c-f9e1-4394-a327-a1eb272d034e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.761074 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639db17c-f9e1-4394-a327-a1eb272d034e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.762789 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 344/680] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:06.768748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa2098-21ca-48fb-8bbd-67eef7cf5e1e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.769109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa2098-21ca-48fb-8bbd-67eef7cf5e1e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.769469 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa2098-21ca-48fb-8bbd-67eef7cf5e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.787798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-60fa2098-21ca-48fb-8bbd-67eef7cf5e1e None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:06.789029 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-60fa2098-21ca-48fb-8bbd-67eef7cf5e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:06.863271 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 337/681] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:07.132401 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639db17c-f9e1-4394-a327-a1eb272d034e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:07.237648 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639db17c-f9e1-4394-a327-a1eb272d034e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.272912 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873618a-4333-4bd0-88ca-f9c78d767c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.273946 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873618a-4333-4bd0-88ca-f9c78d767c79 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.274417 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873618a-4333-4bd0-88ca-f9c78d767c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.274878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873618a-4333-4bd0-88ca-f9c78d767c79 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.286593 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873618a-4333-4bd0-88ca-f9c78d767c79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e873618a-4333-4bd0-88ca-f9c78d767c79 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:07.292050 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:07.293414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:15:07.294454 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:07.297259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 345/682] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:07.303285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa2098-21ca-48fb-8bbd-67eef7cf5e1e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:07.330395 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:07.418749 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa2098-21ca-48fb-8bbd-67eef7cf5e1e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.449094 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b00640-46c4-44c7-8c64-3d928f181587 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.449422 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b00640-46c4-44c7-8c64-3d928f181587 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.449639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b00640-46c4-44c7-8c64-3d928f181587 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.452242 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 346/683] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:07.453680 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2ae03-e691-4993-b640-662fcb30b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.454293 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2ae03-e691-4993-b640-662fcb30b8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.454570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2ae03-e691-4993-b640-662fcb30b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.454903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2ae03-e691-4993-b640-662fcb30b8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.455631 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2ae03-e691-4993-b640-662fcb30b8c5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:07.455939 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2ae03-e691-4993-b640-662fcb30b8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:07.456329 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f2039-4088-40d3-a8d2-51318cc3efbc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.456714 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f2039-4088-40d3-a8d2-51318cc3efbc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.457059 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f2039-4088-40d3-a8d2-51318cc3efbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.457335 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2ae03-e691-4993-b640-662fcb30b8c5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:07.466304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 338/684] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:07.472268 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2ae03-e691-4993-b640-662fcb30b8c5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:07.476555 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9c8f2039-4088-40d3-a8d2-51318cc3efbc None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:07.477857 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9c8f2039-4088-40d3-a8d2-51318cc3efbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:07.503179 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:07.560694 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 347/685] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:07.579272 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:07.596314 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2ae03-e691-4993-b640-662fcb30b8c5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.633101 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2f43b-dbb7-4f2f-8257-f837b89ef05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.633728 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2f43b-dbb7-4f2f-8257-f837b89ef05c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.634007 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2f43b-dbb7-4f2f-8257-f837b89ef05c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.634279 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2f43b-dbb7-4f2f-8257-f837b89ef05c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.645681 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2f43b-dbb7-4f2f-8257-f837b89ef05c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:07.647194 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2f43b-dbb7-4f2f-8257-f837b89ef05c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e6e2f43b-dbb7-4f2f-8257-f837b89ef05c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:07.665511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:07.666444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:07.667309 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:07.667309 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:07.667309 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:07.667309 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 339/686] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:07.672682 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f2039-4088-40d3-a8d2-51318cc3efbc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:07.773686 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f2039-4088-40d3-a8d2-51318cc3efbc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.808360 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4472d-385a-4956-8fb7-63f04cd29492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.809004 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4472d-385a-4956-8fb7-63f04cd29492 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.809281 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4472d-385a-4956-8fb7-63f04cd29492 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.809556 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4472d-385a-4956-8fb7-63f04cd29492 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.810202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4472d-385a-4956-8fb7-63f04cd29492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:07.810503 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4472d-385a-4956-8fb7-63f04cd29492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:07.811861 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4472d-385a-4956-8fb7-63f04cd29492 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:07.823489 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 348/687] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:07.829049 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2f43b-dbb7-4f2f-8257-f837b89ef05c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:07.861778 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4472d-385a-4956-8fb7-63f04cd29492 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:07.888328 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:07.952036 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2f43b-dbb7-4f2f-8257-f837b89ef05c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.985001 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4472d-385a-4956-8fb7-63f04cd29492 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:07.995127 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495964a5-7028-436a-8a2a-341117f5df5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:07.995766 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495964a5-7028-436a-8a2a-341117f5df5a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.996045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495964a5-7028-436a-8a2a-341117f5df5a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.996463 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495964a5-7028-436a-8a2a-341117f5df5a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.005298 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495964a5-7028-436a-8a2a-341117f5df5a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.006739 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495964a5-7028-436a-8a2a-341117f5df5a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.008604 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-495964a5-7028-436a-8a2a-341117f5df5a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:08.022106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:08.024095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:08.025291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:08.025291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:08.025291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:08.025291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:08.025291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:08.025291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:08.025291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:08.025291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:08.025291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:15:08.025291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:08.025995 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe4deb-9a3b-4bcb-98b4-c8f6332384f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.025995 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe4deb-9a3b-4bcb-98b4-c8f6332384f9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.025995 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe4deb-9a3b-4bcb-98b4-c8f6332384f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.025995 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe4deb-9a3b-4bcb-98b4-c8f6332384f9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.026309 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 340/688] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:08.026404 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe4deb-9a3b-4bcb-98b4-c8f6332384f9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.030113 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495964a5-7028-436a-8a2a-341117f5df5a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.030355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe4deb-9a3b-4bcb-98b4-c8f6332384f9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9cbe4deb-9a3b-4bcb-98b4-c8f6332384f9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:08.041210 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:08.042528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:08.043392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 349/689] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:08.049922 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe4deb-9a3b-4bcb-98b4-c8f6332384f9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.152351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.169215 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495964a5-7028-436a-8a2a-341117f5df5a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.170246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe4deb-9a3b-4bcb-98b4-c8f6332384f9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.206727 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a424f68-a852-47f3-b87f-7568e41e5e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.207261 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a424f68-a852-47f3-b87f-7568e41e5e04 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.207540 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a424f68-a852-47f3-b87f-7568e41e5e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.207813 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a424f68-a852-47f3-b87f-7568e41e5e04 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.208444 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a424f68-a852-47f3-b87f-7568e41e5e04 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:08.208834 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a424f68-a852-47f3-b87f-7568e41e5e04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.210214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a424f68-a852-47f3-b87f-7568e41e5e04 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.213841 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50beac16-d176-4522-8e8e-af31ec31bdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.214577 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50beac16-d176-4522-8e8e-af31ec31bdba None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.214863 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50beac16-d176-4522-8e8e-af31ec31bdba None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.215139 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50beac16-d176-4522-8e8e-af31ec31bdba None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.215871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50beac16-d176-4522-8e8e-af31ec31bdba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:08.216178 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50beac16-d176-4522-8e8e-af31ec31bdba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.216363 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 350/690] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:08.218123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50beac16-d176-4522-8e8e-af31ec31bdba None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.223499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a424f68-a852-47f3-b87f-7568e41e5e04 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.242908 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 341/691] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:08.243952 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50beac16-d176-4522-8e8e-af31ec31bdba None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.303467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.353629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a424f68-a852-47f3-b87f-7568e41e5e04 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.354869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50beac16-d176-4522-8e8e-af31ec31bdba None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.402083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d971561-eda6-4c00-8ac9-b6c0e167a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.402774 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d971561-eda6-4c00-8ac9-b6c0e167a743 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.402956 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e80a4c-511b-4597-8e23-c3b4196c2e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.403146 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d971561-eda6-4c00-8ac9-b6c0e167a743 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.403299 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d971561-eda6-4c00-8ac9-b6c0e167a743 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.403476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e80a4c-511b-4597-8e23-c3b4196c2e42 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.403738 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e80a4c-511b-4597-8e23-c3b4196c2e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.403928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d971561-eda6-4c00-8ac9-b6c0e167a743 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:08.404095 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e80a4c-511b-4597-8e23-c3b4196c2e42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.404242 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d971561-eda6-4c00-8ac9-b6c0e167a743 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.405735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d971561-eda6-4c00-8ac9-b6c0e167a743 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.417091 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 351/692] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:08.418346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d971561-eda6-4c00-8ac9-b6c0e167a743 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.436704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.479915 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e80a4c-511b-4597-8e23-c3b4196c2e42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.481449 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e80a4c-511b-4597-8e23-c3b4196c2e42 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.529895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d971561-eda6-4c00-8ac9-b6c0e167a743 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.539523 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.567232 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf0d87-af4b-4430-ad69-f392c8e97882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.567846 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf0d87-af4b-4430-ad69-f392c8e97882 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.568126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf0d87-af4b-4430-ad69-f392c8e97882 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.568393 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf0d87-af4b-4430-ad69-f392c8e97882 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.569139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf0d87-af4b-4430-ad69-f392c8e97882 None admin] RBAC: Adding query filter params (interface=public, service_id=1d80694b2f574a51a030301caaf710f7, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:08.569452 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf0d87-af4b-4430-ad69-f392c8e97882 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.570811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf0d87-af4b-4430-ad69-f392c8e97882 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.580305 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 352/693] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/endpoints?service_id=1d80694b2f574a51a030301caaf710f7&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:08.580305 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d108469-0ab9-4911-9c6e-42f558f3e27e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.580305 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d108469-0ab9-4911-9c6e-42f558f3e27e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.580305 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d108469-0ab9-4911-9c6e-42f558f3e27e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.580305 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 353/694] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:08.583636 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d108469-0ab9-4911-9c6e-42f558f3e27e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.598532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 342/695] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 07:15:08.598532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fb43d-d33a-477b-9652-077b216e1fa0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.598532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fb43d-d33a-477b-9652-077b216e1fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.598532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fb43d-d33a-477b-9652-077b216e1fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.614925 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-288fb43d-d33a-477b-9652-077b216e1fa0 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:08.616228 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-288fb43d-d33a-477b-9652-077b216e1fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:08.680166 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.690638 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 343/696] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:08.691689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fb43d-d33a-477b-9652-077b216e1fa0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.716322 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d108469-0ab9-4911-9c6e-42f558f3e27e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.757644 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678b245-7dd7-46fd-8646-376ac2df7c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.758347 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678b245-7dd7-46fd-8646-376ac2df7c5b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.758717 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678b245-7dd7-46fd-8646-376ac2df7c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.759107 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678b245-7dd7-46fd-8646-376ac2df7c5b None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.759974 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678b245-7dd7-46fd-8646-376ac2df7c5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.761794 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678b245-7dd7-46fd-8646-376ac2df7c5b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.780132 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 354/697] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/services => generated 1497 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:15:08.797923 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.817905 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fb43d-d33a-477b-9652-077b216e1fa0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.843220 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6523de-8139-46d7-9bd5-90aa533ac199 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.843855 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6523de-8139-46d7-9bd5-90aa533ac199 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.844332 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6523de-8139-46d7-9bd5-90aa533ac199 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.846258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 355/698] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:08.851720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2bc77-0c4f-461e-aeeb-bbc3ba7bbf39 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.851720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2bc77-0c4f-461e-aeeb-bbc3ba7bbf39 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.851992 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2bc77-0c4f-461e-aeeb-bbc3ba7bbf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.858756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa84ba4-e57e-41ad-b57f-357337bb2f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.859665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa84ba4-e57e-41ad-b57f-357337bb2f6b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.860166 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa84ba4-e57e-41ad-b57f-357337bb2f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.860665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa84ba4-e57e-41ad-b57f-357337bb2f6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.873188 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa84ba4-e57e-41ad-b57f-357337bb2f6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.874913 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa84ba4-e57e-41ad-b57f-357337bb2f6b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.877148 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-70e2bc77-0c4f-461e-aeeb-bbc3ba7bbf39 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:08.878475 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-70e2bc77-0c4f-461e-aeeb-bbc3ba7bbf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1aa84ba4-e57e-41ad-b57f-357337bb2f6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:08.892871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:08.894075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:08.895169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:08.895169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:08.895169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:08.897136 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 344/699] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:08.903975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa84ba4-e57e-41ad-b57f-357337bb2f6b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.969949 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 356/700] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:08.972753 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2bc77-0c4f-461e-aeeb-bbc3ba7bbf39 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.028889 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa84ba4-e57e-41ad-b57f-357337bb2f6b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.056473 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.069255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ca017-175e-47ca-a7b0-425d8e62e5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.070426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ca017-175e-47ca-a7b0-425d8e62e5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.070869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ca017-175e-47ca-a7b0-425d8e62e5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.071544 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ca017-175e-47ca-a7b0-425d8e62e5f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.072845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ca017-175e-47ca-a7b0-425d8e62e5f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:09.073599 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ca017-175e-47ca-a7b0-425d8e62e5f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.075391 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ca017-175e-47ca-a7b0-425d8e62e5f3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.083978 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.090860 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 345/701] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:09.093650 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2bc77-0c4f-461e-aeeb-bbc3ba7bbf39 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.096688 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ca017-175e-47ca-a7b0-425d8e62e5f3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.132841 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b9270-43b2-4c06-94f5-735730341de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.133478 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b9270-43b2-4c06-94f5-735730341de0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.133921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b9270-43b2-4c06-94f5-735730341de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.134344 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b9270-43b2-4c06-94f5-735730341de0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.135078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b9270-43b2-4c06-94f5-735730341de0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.136620 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b9270-43b2-4c06-94f5-735730341de0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-aa4b9270-43b2-4c06-94f5-735730341de0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:09.146833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:09.147824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:09.149176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:09.152633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 357/702] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:09.154475 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b9270-43b2-4c06-94f5-735730341de0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.212830 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ca017-175e-47ca-a7b0-425d8e62e5f3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.250085 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aaa77-0b46-442f-8822-3e05ea27ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.250732 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aaa77-0b46-442f-8822-3e05ea27ec09 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.251008 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aaa77-0b46-442f-8822-3e05ea27ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.251281 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aaa77-0b46-442f-8822-3e05ea27ec09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.260254 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123aaa77-0b46-442f-8822-3e05ea27ec09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.262228 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123aaa77-0b46-442f-8822-3e05ea27ec09 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.268751 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b9270-43b2-4c06-94f5-735730341de0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-123aaa77-0b46-442f-8822-3e05ea27ec09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:09.274097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:09.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:09.280654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:09.280654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:09.280654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:09.280654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:09.280654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:09.280654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:09.280654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:09.280654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:09.280654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 346/703] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:09.280654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aaa77-0b46-442f-8822-3e05ea27ec09 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.310968 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5058958-c4ec-4c49-82e5-a6916e2847ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.311666 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5058958-c4ec-4c49-82e5-a6916e2847ec None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.311946 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5058958-c4ec-4c49-82e5-a6916e2847ec None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.312213 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5058958-c4ec-4c49-82e5-a6916e2847ec None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.312871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5058958-c4ec-4c49-82e5-a6916e2847ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:09.313170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5058958-c4ec-4c49-82e5-a6916e2847ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.314581 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5058958-c4ec-4c49-82e5-a6916e2847ec None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.316367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.321244 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 358/704] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.322054 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5058958-c4ec-4c49-82e5-a6916e2847ec None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.339520 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.382484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aaa77-0b46-442f-8822-3e05ea27ec09 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.416768 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5b85f-5b81-4feb-ad78-6e2384f4ad0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.417399 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b85f-5b81-4feb-ad78-6e2384f4ad0d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.417692 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b85f-5b81-4feb-ad78-6e2384f4ad0d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.417955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b85f-5b81-4feb-ad78-6e2384f4ad0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.418535 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5b85f-5b81-4feb-ad78-6e2384f4ad0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.419949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5b85f-5b81-4feb-ad78-6e2384f4ad0d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.428359 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5058958-c4ec-4c49-82e5-a6916e2847ec None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c4a5b85f-5b81-4feb-ad78-6e2384f4ad0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:09.428949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:09.429903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:09.430769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 347/705] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:09.431585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5b85f-5b81-4feb-ad78-6e2384f4ad0d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.468126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2c5f8-d4ad-4b1e-a544-31fca08ffe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.468769 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c5f8-d4ad-4b1e-a544-31fca08ffe8f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.469044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c5f8-d4ad-4b1e-a544-31fca08ffe8f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.469313 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c5f8-d4ad-4b1e-a544-31fca08ffe8f None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.470018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2c5f8-d4ad-4b1e-a544-31fca08ffe8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:09.470829 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2c5f8-d4ad-4b1e-a544-31fca08ffe8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.472257 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2c5f8-d4ad-4b1e-a544-31fca08ffe8f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.486713 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 359/706] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.487547 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2c5f8-d4ad-4b1e-a544-31fca08ffe8f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.540955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5b85f-5b81-4feb-ad78-6e2384f4ad0d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.547595 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.568411 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.581229 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79e59d-7283-4c91-81ea-4b9045eee726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.581850 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79e59d-7283-4c91-81ea-4b9045eee726 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.582123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79e59d-7283-4c91-81ea-4b9045eee726 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.582399 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79e59d-7283-4c91-81ea-4b9045eee726 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.583089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79e59d-7283-4c91-81ea-4b9045eee726 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:09.583326 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79e59d-7283-4c91-81ea-4b9045eee726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.584718 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79e59d-7283-4c91-81ea-4b9045eee726 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.589923 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 348/707] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.593696 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2c5f8-d4ad-4b1e-a544-31fca08ffe8f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.593822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79e59d-7283-4c91-81ea-4b9045eee726 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.632792 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f12008-e2b8-42d7-abcb-d747e02f9160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.633924 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f12008-e2b8-42d7-abcb-d747e02f9160 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.634205 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f12008-e2b8-42d7-abcb-d747e02f9160 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.634476 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f12008-e2b8-42d7-abcb-d747e02f9160 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.635108 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f12008-e2b8-42d7-abcb-d747e02f9160 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:09.638541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f12008-e2b8-42d7-abcb-d747e02f9160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.638541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f12008-e2b8-42d7-abcb-d747e02f9160 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.646953 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 360/708] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.647616 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f12008-e2b8-42d7-abcb-d747e02f9160 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.703124 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79e59d-7283-4c91-81ea-4b9045eee726 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.743154 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf99ab-296e-44ed-b3f2-079b582c830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.744114 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf99ab-296e-44ed-b3f2-079b582c830a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.744397 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf99ab-296e-44ed-b3f2-079b582c830a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.744705 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf99ab-296e-44ed-b3f2-079b582c830a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.745428 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf99ab-296e-44ed-b3f2-079b582c830a None admin] RBAC: Adding query filter params (role.id=d660bf006ef649159f7369c7f1623c04, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=b67332ed32044f94a1b3a6a6e6bbb348) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:09.745735 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf99ab-296e-44ed-b3f2-079b582c830a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.747156 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf99ab-296e-44ed-b3f2-079b582c830a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.753306 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 349/709] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/role_assignments?user.id=b67332ed32044f94a1b3a6a6e6bbb348&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=d660bf006ef649159f7369c7f1623c04 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.753919 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf99ab-296e-44ed-b3f2-079b582c830a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.764429 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f12008-e2b8-42d7-abcb-d747e02f9160 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.801922 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.813651 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.822800 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ffbe8-53be-46d1-a6c0-4ee965f6950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.824049 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ffbe8-53be-46d1-a6c0-4ee965f6950e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.824754 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ffbe8-53be-46d1-a6c0-4ee965f6950e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.825058 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ffbe8-53be-46d1-a6c0-4ee965f6950e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.825804 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ffbe8-53be-46d1-a6c0-4ee965f6950e None admin] RBAC: Adding query filter params (interface=public, service_id=83209b40d5d6451b977482a3359b03e4, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:09.826121 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ffbe8-53be-46d1-a6c0-4ee965f6950e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.827520 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ffbe8-53be-46d1-a6c0-4ee965f6950e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.839575 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 361/710] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/endpoints?service_id=83209b40d5d6451b977482a3359b03e4&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.840585 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ffbe8-53be-46d1-a6c0-4ee965f6950e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.889132 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf99ab-296e-44ed-b3f2-079b582c830a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.928212 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1586d7c-598d-4ef2-8397-49c0c126e26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.928949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1586d7c-598d-4ef2-8397-49c0c126e26d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.929297 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1586d7c-598d-4ef2-8397-49c0c126e26d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.929641 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1586d7c-598d-4ef2-8397-49c0c126e26d None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.930404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1586d7c-598d-4ef2-8397-49c0c126e26d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:09.930701 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1586d7c-598d-4ef2-8397-49c0c126e26d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.933174 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1586d7c-598d-4ef2-8397-49c0c126e26d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.939008 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 350/711] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.975693 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ffbe8-53be-46d1-a6c0-4ee965f6950e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.015239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6996d-6c8e-4782-9263-25d40e70a63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.015866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6996d-6c8e-4782-9263-25d40e70a63c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.016141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6996d-6c8e-4782-9263-25d40e70a63c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.016410 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6996d-6c8e-4782-9263-25d40e70a63c None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.017585 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6996d-6c8e-4782-9263-25d40e70a63c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.019045 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6996d-6c8e-4782-9263-25d40e70a63c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.026397 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 362/712] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/services => generated 1497 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:15:11.022634 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878be818-e4c1-426a-9efe-98625439ae62 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.023112 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878be818-e4c1-426a-9efe-98625439ae62 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.023112 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878be818-e4c1-426a-9efe-98625439ae62 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.024748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 351/713] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:11.030002 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607fc0d-8e54-4e1f-ae2f-dce14a8c4ca0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.030390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607fc0d-8e54-4e1f-ae2f-dce14a8c4ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.030554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607fc0d-8e54-4e1f-ae2f-dce14a8c4ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.049775 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2607fc0d-8e54-4e1f-ae2f-dce14a8c4ca0 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:11.050947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2607fc0d-8e54-4e1f-ae2f-dce14a8c4ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:11.099637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:11.127840 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 363/714] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:11.389244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878be818-e4c1-426a-9efe-98625439ae62 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:11.491717 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878be818-e4c1-426a-9efe-98625439ae62 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:11.506548 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:11.528400 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fa10c-1099-4dde-8735-58d73cd95fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:11.529402 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fa10c-1099-4dde-8735-58d73cd95fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.529889 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fa10c-1099-4dde-8735-58d73cd95fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.530365 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fa10c-1099-4dde-8735-58d73cd95fb9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.531185 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fa10c-1099-4dde-8735-58d73cd95fb9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:11.533364 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fa10c-1099-4dde-8735-58d73cd95fb9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c97fa10c-1099-4dde-8735-58d73cd95fb9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:11.542710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:11.543724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:15:11.544565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:11.547266 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 352/715] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:11.553064 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607fc0d-8e54-4e1f-ae2f-dce14a8c4ca0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:11.653637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607fc0d-8e54-4e1f-ae2f-dce14a8c4ca0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:11.687653 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b8cac-a8b0-46e4-9782-56f7ba2aaafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:11.688270 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b8cac-a8b0-46e4-9782-56f7ba2aaafe None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.688564 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b8cac-a8b0-46e4-9782-56f7ba2aaafe None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.688843 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b8cac-a8b0-46e4-9782-56f7ba2aaafe None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.690008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b8cac-a8b0-46e4-9782-56f7ba2aaafe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:11.690309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b8cac-a8b0-46e4-9782-56f7ba2aaafe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:11.691667 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b8cac-a8b0-46e4-9782-56f7ba2aaafe None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:11.700649 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 364/716] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:11.702581 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fa10c-1099-4dde-8735-58d73cd95fb9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:11.803905 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fa10c-1099-4dde-8735-58d73cd95fb9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:11.838669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43982d7d-89fb-4681-a3b9-b6c01a34ce5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:11.839317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43982d7d-89fb-4681-a3b9-b6c01a34ce5e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.839598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43982d7d-89fb-4681-a3b9-b6c01a34ce5e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.839866 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43982d7d-89fb-4681-a3b9-b6c01a34ce5e None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.840563 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43982d7d-89fb-4681-a3b9-b6c01a34ce5e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:11.840902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43982d7d-89fb-4681-a3b9-b6c01a34ce5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:11.842312 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43982d7d-89fb-4681-a3b9-b6c01a34ce5e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:11.852394 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 353/717] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:11.853733 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b8cac-a8b0-46e4-9782-56f7ba2aaafe None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:11.901489 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:11.957842 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b8cac-a8b0-46e4-9782-56f7ba2aaafe None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:11.997406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6652bf-b257-4ddb-a6d7-9185f7000905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:11.998026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6652bf-b257-4ddb-a6d7-9185f7000905 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.998301 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6652bf-b257-4ddb-a6d7-9185f7000905 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.998569 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6652bf-b257-4ddb-a6d7-9185f7000905 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.999251 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6652bf-b257-4ddb-a6d7-9185f7000905 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.000716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6652bf-b257-4ddb-a6d7-9185f7000905 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.028737 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 365/718] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Nov 2 07:15:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:12.097962 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c424b-a96e-42e4-9bf8-af9995825819 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.098493 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c424b-a96e-42e4-9bf8-af9995825819 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.098493 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c424b-a96e-42e4-9bf8-af9995825819 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.100014 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 354/719] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:12.106000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5456e-b607-4fd6-b31d-4489f74e1ba5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.106976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5456e-b607-4fd6-b31d-4489f74e1ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.107458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5456e-b607-4fd6-b31d-4489f74e1ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.132022 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ffa5456e-b607-4fd6-b31d-4489f74e1ba5 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:12.133263 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ffa5456e-b607-4fd6-b31d-4489f74e1ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:12.136518 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a3a42-6e4f-4b44-accd-586897954665 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.136827 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a3a42-6e4f-4b44-accd-586897954665 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.137106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a3a42-6e4f-4b44-accd-586897954665 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.138772 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 355/720] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:12.145330 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ee408-9cb5-4761-805d-a881d239db89 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.145644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ee408-9cb5-4761-805d-a881d239db89 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.145891 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ee408-9cb5-4761-805d-a881d239db89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.165396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fd5ee408-9cb5-4761-805d-a881d239db89 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:12.166670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fd5ee408-9cb5-4761-805d-a881d239db89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:12.210344 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:12.240966 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 366/721] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:12.253697 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 356/722] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:12.277220 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54228b8-d41f-4f6e-bed6-6b06b3d8dc9f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.277472 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54228b8-d41f-4f6e-bed6-6b06b3d8dc9f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.277730 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54228b8-d41f-4f6e-bed6-6b06b3d8dc9f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.279213 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 367/723] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:12.284585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc17824-0520-43f3-8a88-f6fce20d933b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.284865 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc17824-0520-43f3-8a88-f6fce20d933b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.285139 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc17824-0520-43f3-8a88-f6fce20d933b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.305139 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5cc17824-0520-43f3-8a88-f6fce20d933b None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:12.306314 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5cc17824-0520-43f3-8a88-f6fce20d933b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:12.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 357/724] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:12.513472 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54228b8-d41f-4f6e-bed6-6b06b3d8dc9f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.542577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:12.554137 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc17824-0520-43f3-8a88-f6fce20d933b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.633015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54228b8-d41f-4f6e-bed6-6b06b3d8dc9f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.683423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc17824-0520-43f3-8a88-f6fce20d933b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.684450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb96b7e-09b5-4623-852a-73b8257d271b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.685008 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:12.685857 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb96b7e-09b5-4623-852a-73b8257d271b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.686731 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb96b7e-09b5-4623-852a-73b8257d271b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.686731 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb96b7e-09b5-4623-852a-73b8257d271b None admin] PATH_INFO: `/v3/services/image` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.687021 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb96b7e-09b5-4623-852a-73b8257d271b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.689651 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb96b7e-09b5-4623-852a-73b8257d271b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-fdb96b7e-09b5-4623-852a-73b8257d271b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:12.699383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:12.700372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:15:12.701541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:12.705202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 368/725] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Nov 2 07:15:12 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:12.706236 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb96b7e-09b5-4623-852a-73b8257d271b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.727118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ab762-90b9-454e-a946-fc521f46238b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.728058 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ab762-90b9-454e-a946-fc521f46238b None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.728578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ab762-90b9-454e-a946-fc521f46238b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.729470 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ab762-90b9-454e-a946-fc521f46238b None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.730644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ab762-90b9-454e-a946-fc521f46238b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.732831 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ab762-90b9-454e-a946-fc521f46238b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.761455 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 358/726] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:15:12 2022] POST /identity/v3/services => generated 242 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:12.761455 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ab762-90b9-454e-a946-fc521f46238b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.828857 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:12.843054 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb96b7e-09b5-4623-852a-73b8257d271b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.899736 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ab762-90b9-454e-a946-fc521f46238b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.899991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f82781-a4c4-49c5-a9b4-713650ccb3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.900384 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f82781-a4c4-49c5-a9b4-713650ccb3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.900771 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f82781-a4c4-49c5-a9b4-713650ccb3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.901042 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f82781-a4c4-49c5-a9b4-713650ccb3d1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.901633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f82781-a4c4-49c5-a9b4-713650ccb3d1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.903382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f82781-a4c4-49c5-a9b4-713650ccb3d1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a8f82781-a4c4-49c5-a9b4-713650ccb3d1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:12.921480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:12.922770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:12.924761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 369/727] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Nov 2 07:15:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:12.931626 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f82781-a4c4-49c5-a9b4-713650ccb3d1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.948693 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c2f34-3714-4c65-a57b-9aa9422aefdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.949241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c2f34-3714-4c65-a57b-9aa9422aefdf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.949672 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c2f34-3714-4c65-a57b-9aa9422aefdf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.951948 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c2f34-3714-4c65-a57b-9aa9422aefdf None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.951948 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c2f34-3714-4c65-a57b-9aa9422aefdf None admin] RBAC: Adding query filter params (name=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:12.951948 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c2f34-3714-4c65-a57b-9aa9422aefdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.953023 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c2f34-3714-4c65-a57b-9aa9422aefdf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.960708 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 359/728] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:12.966785 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c2f34-3714-4c65-a57b-9aa9422aefdf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.994770 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.060961 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f82781-a4c4-49c5-a9b4-713650ccb3d1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.099090 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5003cc9-527a-4d94-92d3-419cfd0c9b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.100035 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5003cc9-527a-4d94-92d3-419cfd0c9b3c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.100514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c2f34-3714-4c65-a57b-9aa9422aefdf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.101148 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5003cc9-527a-4d94-92d3-419cfd0c9b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.101636 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5003cc9-527a-4d94-92d3-419cfd0c9b3c None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.102625 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5003cc9-527a-4d94-92d3-419cfd0c9b3c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:13.103221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5003cc9-527a-4d94-92d3-419cfd0c9b3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.104901 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5003cc9-527a-4d94-92d3-419cfd0c9b3c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.111012 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 370/729] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:13.117452 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5003cc9-527a-4d94-92d3-419cfd0c9b3c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.137629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.139781 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781f8a6-2974-489b-ac4b-77822b4a6003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.140483 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781f8a6-2974-489b-ac4b-77822b4a6003 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.140797 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781f8a6-2974-489b-ac4b-77822b4a6003 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.141203 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781f8a6-2974-489b-ac4b-77822b4a6003 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.141822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781f8a6-2974-489b-ac4b-77822b4a6003 None admin] RBAC: Adding query filter params (type=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:13.142185 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781f8a6-2974-489b-ac4b-77822b4a6003 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.144703 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781f8a6-2974-489b-ac4b-77822b4a6003 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.156649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 360/730] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:13.243240 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5003cc9-527a-4d94-92d3-419cfd0c9b3c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.281414 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f6087-161c-40b7-adcf-f1007c900097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.282055 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f6087-161c-40b7-adcf-f1007c900097 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.282333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f6087-161c-40b7-adcf-f1007c900097 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.283128 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f6087-161c-40b7-adcf-f1007c900097 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.283828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f6087-161c-40b7-adcf-f1007c900097 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:13.284141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f6087-161c-40b7-adcf-f1007c900097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.285598 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f6087-161c-40b7-adcf-f1007c900097 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.291539 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 371/731] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:13.298346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781f8a6-2974-489b-ac4b-77822b4a6003 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.395317 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.412728 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781f8a6-2974-489b-ac4b-77822b4a6003 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.448310 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4ccf-ea89-405e-894b-b07b3620c687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.448957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ccf-ea89-405e-894b-b07b3620c687 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.449237 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ccf-ea89-405e-894b-b07b3620c687 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.449515 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ccf-ea89-405e-894b-b07b3620c687 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.450185 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4ccf-ea89-405e-894b-b07b3620c687 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.451537 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4ccf-ea89-405e-894b-b07b3620c687 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.474290 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 361/732] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Nov 2 07:15:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:14.317408 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ccf69-de37-4351-9018-ecdcdb69172b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:14.317928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ccf69-de37-4351-9018-ecdcdb69172b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:14.317928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ccf69-de37-4351-9018-ecdcdb69172b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:14.319528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 372/733] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:14.324878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2b907-b135-4c17-a0f6-6b7bab2f24db None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:14.325285 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2b907-b135-4c17-a0f6-6b7bab2f24db None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:14.325454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2b907-b135-4c17-a0f6-6b7bab2f24db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:14.344010 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-13d2b907-b135-4c17-a0f6-6b7bab2f24db None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:14.345248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-13d2b907-b135-4c17-a0f6-6b7bab2f24db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:14.429210 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 362/734] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:14.704739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ccf69-de37-4351-9018-ecdcdb69172b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:14.788330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:14.812918 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ccf69-de37-4351-9018-ecdcdb69172b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:14.852752 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4585c1-2622-41ef-95ba-8716084f118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:14.853714 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4585c1-2622-41ef-95ba-8716084f118d None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:14.854251 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4585c1-2622-41ef-95ba-8716084f118d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:14.854731 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4585c1-2622-41ef-95ba-8716084f118d None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:14.855588 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4585c1-2622-41ef-95ba-8716084f118d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:14.857237 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4585c1-2622-41ef-95ba-8716084f118d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:14.860046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: WARNING keystone.server.flask.common [None req-2d4585c1-2622-41ef-95ba-8716084f118d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Nov 02 07:15:14.879581 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.sql.core [None req-2d4585c1-2622-41ef-95ba-8716084f118d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:15:14.879581 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:15:14.879581 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [parameters: {'id': 'bbfe318941c346b4afda8f93377e4abd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 07:15:14.879581 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81400) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2d4585c1-2622-41ef-95ba-8716084f118d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbfe318941c346b4afda8f93377e4abd.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbfe318941c346b4afda8f93377e4abd. Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:15:14.888171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:15:14.889015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application yield resource Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise self.value Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application session.commit() Nov 02 07:15:14.889976 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:15:14.890928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:15:14.892716 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:15:14.893554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [parameters: {'id': 'bbfe318941c346b4afda8f93377e4abd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:14.894436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:14.895233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:14.896207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:14.896207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Nov 02 07:15:14.896207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Nov 02 07:15:14.896207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:14.896207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:14.896207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:15:14.896207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:15:14.896207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbfe318941c346b4afda8f93377e4abd. Nov 02 07:15:14.896207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:14.896207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 373/735] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Nov 2 07:15:14 2022] POST /identity/v3/projects => generated 262 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:14.899926 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2b907-b135-4c17-a0f6-6b7bab2f24db None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:14.974833 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.015425 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2b907-b135-4c17-a0f6-6b7bab2f24db None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.051888 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca0db9-2d9b-4a37-86c0-b63dd9bbb201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.052602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca0db9-2d9b-4a37-86c0-b63dd9bbb201 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.052886 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca0db9-2d9b-4a37-86c0-b63dd9bbb201 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.053155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca0db9-2d9b-4a37-86c0-b63dd9bbb201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.060368 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca0db9-2d9b-4a37-86c0-b63dd9bbb201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.061830 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca0db9-2d9b-4a37-86c0-b63dd9bbb201 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-23ca0db9-2d9b-4a37-86c0-b63dd9bbb201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:15.075346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:15.076262 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:15.077181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:15.077181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:15.077181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:15.077181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:15.077181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:15.077181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:15.077181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:15.077181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:15.080088 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 363/736] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:15.085897 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4585c1-2622-41ef-95ba-8716084f118d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.096991 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec009a-216d-4c62-8e7a-6424e27e6c41 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.097555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec009a-216d-4c62-8e7a-6424e27e6c41 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.098001 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec009a-216d-4c62-8e7a-6424e27e6c41 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.099756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 364/737] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:15.104759 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5996042-0adc-4df9-881c-0da327c3529e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.105213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5996042-0adc-4df9-881c-0da327c3529e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.105638 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5996042-0adc-4df9-881c-0da327c3529e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.124263 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c5996042-0adc-4df9-881c-0da327c3529e None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:15.125693 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c5996042-0adc-4df9-881c-0da327c3529e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:15.208912 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 365/738] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:15.209098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4585c1-2622-41ef-95ba-8716084f118d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.248645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaf095-fe57-42ac-97fb-a6d3152529bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.250164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaf095-fe57-42ac-97fb-a6d3152529bf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.250838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaf095-fe57-42ac-97fb-a6d3152529bf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.251794 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaf095-fe57-42ac-97fb-a6d3152529bf None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.253008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaf095-fe57-42ac-97fb-a6d3152529bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:15.253529 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaf095-fe57-42ac-97fb-a6d3152529bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.255423 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaf095-fe57-42ac-97fb-a6d3152529bf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.257497 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.279857 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 374/739] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:15.322721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66c16a-fbd6-454d-b2aa-619a2ac60e1c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.323054 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66c16a-fbd6-454d-b2aa-619a2ac60e1c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.323271 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66c16a-fbd6-454d-b2aa-619a2ac60e1c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.324939 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 366/740] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:15.331523 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2b8ce-4f89-4c07-aa91-e566552e4301 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.332024 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2b8ce-4f89-4c07-aa91-e566552e4301 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.332099 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2b8ce-4f89-4c07-aa91-e566552e4301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.352611 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1af2b8ce-4f89-4c07-aa91-e566552e4301 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:15.354342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1af2b8ce-4f89-4c07-aa91-e566552e4301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:15.432024 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 375/741] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:15.486836 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66c16a-fbd6-454d-b2aa-619a2ac60e1c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.550690 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.598091 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66c16a-fbd6-454d-b2aa-619a2ac60e1c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.636343 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0e25a-6c7a-4a7c-bdb7-431045360c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.637055 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0e25a-6c7a-4a7c-bdb7-431045360c01 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.637329 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0e25a-6c7a-4a7c-bdb7-431045360c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.637598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0e25a-6c7a-4a7c-bdb7-431045360c01 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.638174 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0e25a-6c7a-4a7c-bdb7-431045360c01 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.639583 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0e25a-6c7a-4a7c-bdb7-431045360c01 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-20b0e25a-6c7a-4a7c-bdb7-431045360c01 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:15.649624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:15.650623 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:15.651444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 367/742] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:15.656898 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2b8ce-4f89-4c07-aa91-e566552e4301 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.703729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0e25a-6c7a-4a7c-bdb7-431045360c01 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.764523 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2b8ce-4f89-4c07-aa91-e566552e4301 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.789933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.802486 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78f6f0-66e6-4c40-be19-a969ff4c8af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.803106 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78f6f0-66e6-4c40-be19-a969ff4c8af9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.803373 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78f6f0-66e6-4c40-be19-a969ff4c8af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.803671 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78f6f0-66e6-4c40-be19-a969ff4c8af9 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.804270 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78f6f0-66e6-4c40-be19-a969ff4c8af9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:15.804593 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78f6f0-66e6-4c40-be19-a969ff4c8af9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.805955 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78f6f0-66e6-4c40-be19-a969ff4c8af9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.810621 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0e25a-6c7a-4a7c-bdb7-431045360c01 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.811268 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 376/743] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:15.812498 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86e14e-8bfb-4326-8ad3-133ab53d3cec None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.812803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86e14e-8bfb-4326-8ad3-133ab53d3cec None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.813071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86e14e-8bfb-4326-8ad3-133ab53d3cec None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.814471 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 377/744] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:15.817394 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86e14e-8bfb-4326-8ad3-133ab53d3cec None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.838897 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.851496 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462d64c-0966-46a7-87e0-b2aa4ab2f938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.852112 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462d64c-0966-46a7-87e0-b2aa4ab2f938 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.852395 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462d64c-0966-46a7-87e0-b2aa4ab2f938 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.852707 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462d64c-0966-46a7-87e0-b2aa4ab2f938 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.853375 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462d64c-0966-46a7-87e0-b2aa4ab2f938 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.854735 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462d64c-0966-46a7-87e0-b2aa4ab2f938 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.870601 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 368/745] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:15:15 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:15.871888 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16847895-d7c5-447e-b96f-76eec96aa91f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.872168 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16847895-d7c5-447e-b96f-76eec96aa91f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.872436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16847895-d7c5-447e-b96f-76eec96aa91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.892670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-16847895-d7c5-447e-b96f-76eec96aa91f None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:15.894287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-16847895-d7c5-447e-b96f-76eec96aa91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:15.959516 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86e14e-8bfb-4326-8ad3-133ab53d3cec None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.973872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 369/746] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:15 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:16.004349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b9e65-494e-4aff-9b89-3737d5ef0768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.005357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9e65-494e-4aff-9b89-3737d5ef0768 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.005816 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9e65-494e-4aff-9b89-3737d5ef0768 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.006288 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9e65-494e-4aff-9b89-3737d5ef0768 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.007221 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b9e65-494e-4aff-9b89-3737d5ef0768 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:16.007704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b9e65-494e-4aff-9b89-3737d5ef0768 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.009345 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b9e65-494e-4aff-9b89-3737d5ef0768 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.015678 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 378/747] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:16.022611 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16847895-d7c5-447e-b96f-76eec96aa91f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.137605 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16847895-d7c5-447e-b96f-76eec96aa91f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.152273 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:16.176316 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f3f9b-4cca-4f9a-a272-07aa8a85bfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.176961 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f3f9b-4cca-4f9a-a272-07aa8a85bfd7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.177240 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f3f9b-4cca-4f9a-a272-07aa8a85bfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.177511 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f3f9b-4cca-4f9a-a272-07aa8a85bfd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.178264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f3f9b-4cca-4f9a-a272-07aa8a85bfd7 None admin] RBAC: Adding query filter params (interface=public, service_id=d529041fd5f84a55b69f294481306418, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:16.178573 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f3f9b-4cca-4f9a-a272-07aa8a85bfd7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.179935 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f3f9b-4cca-4f9a-a272-07aa8a85bfd7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.185759 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 370/748] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/endpoints?service_id=d529041fd5f84a55b69f294481306418&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:16.191896 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b9e65-494e-4aff-9b89-3737d5ef0768 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.246466 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:16.274241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f3f9b-4cca-4f9a-a272-07aa8a85bfd7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.303340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b9e65-494e-4aff-9b89-3737d5ef0768 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.342443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628b74c-1fdb-469f-8060-1019bcb5c825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.343133 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628b74c-1fdb-469f-8060-1019bcb5c825 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.343424 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628b74c-1fdb-469f-8060-1019bcb5c825 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.343627 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628b74c-1fdb-469f-8060-1019bcb5c825 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.344204 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628b74c-1fdb-469f-8060-1019bcb5c825 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.346205 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628b74c-1fdb-469f-8060-1019bcb5c825 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.354379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 379/749] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:15:16.405861 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f3f9b-4cca-4f9a-a272-07aa8a85bfd7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.446781 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54269a0b-18bb-4286-b23d-c5900dbbd0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.447808 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54269a0b-18bb-4286-b23d-c5900dbbd0fa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.448334 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54269a0b-18bb-4286-b23d-c5900dbbd0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.448860 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54269a0b-18bb-4286-b23d-c5900dbbd0fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.449742 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54269a0b-18bb-4286-b23d-c5900dbbd0fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.451406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54269a0b-18bb-4286-b23d-c5900dbbd0fa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-54269a0b-18bb-4286-b23d-c5900dbbd0fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:16.461685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:16.462979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:16.463845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:16.472077 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 371/750] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:16.473809 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628b74c-1fdb-469f-8060-1019bcb5c825 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.582559 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628b74c-1fdb-469f-8060-1019bcb5c825 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.597082 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:16.620016 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af51d0f-2fc9-42c5-8c4b-b573ca920e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.620661 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af51d0f-2fc9-42c5-8c4b-b573ca920e50 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.620939 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af51d0f-2fc9-42c5-8c4b-b573ca920e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.621206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af51d0f-2fc9-42c5-8c4b-b573ca920e50 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.621844 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af51d0f-2fc9-42c5-8c4b-b573ca920e50 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:16.622140 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af51d0f-2fc9-42c5-8c4b-b573ca920e50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.623490 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af51d0f-2fc9-42c5-8c4b-b573ca920e50 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.632656 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 380/751] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:16.634883 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54269a0b-18bb-4286-b23d-c5900dbbd0fa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.682137 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:16.746019 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54269a0b-18bb-4286-b23d-c5900dbbd0fa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.781598 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25706e0-476b-4fae-9bba-47ef79dd50cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.782213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25706e0-476b-4fae-9bba-47ef79dd50cd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.782488 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25706e0-476b-4fae-9bba-47ef79dd50cd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.782818 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25706e0-476b-4fae-9bba-47ef79dd50cd None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.783440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25706e0-476b-4fae-9bba-47ef79dd50cd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:16.783735 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25706e0-476b-4fae-9bba-47ef79dd50cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.785117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25706e0-476b-4fae-9bba-47ef79dd50cd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.795104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 372/752] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:16.796599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af51d0f-2fc9-42c5-8c4b-b573ca920e50 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.907212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af51d0f-2fc9-42c5-8c4b-b573ca920e50 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.944471 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50579a2-d173-4adb-97aa-8ddc55f85fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.945112 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50579a2-d173-4adb-97aa-8ddc55f85fcd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.945390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50579a2-d173-4adb-97aa-8ddc55f85fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.945721 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50579a2-d173-4adb-97aa-8ddc55f85fcd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.946425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50579a2-d173-4adb-97aa-8ddc55f85fcd None admin] RBAC: Adding query filter params (interface=admin, service_id=83209b40d5d6451b977482a3359b03e4, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:16.946731 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50579a2-d173-4adb-97aa-8ddc55f85fcd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.948095 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50579a2-d173-4adb-97aa-8ddc55f85fcd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.953698 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 381/753] 10.222.0.15 () {56 vars in 1249 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/endpoints?service_id=83209b40d5d6451b977482a3359b03e4&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:16.959468 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25706e0-476b-4fae-9bba-47ef79dd50cd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:17.068684 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25706e0-476b-4fae-9bba-47ef79dd50cd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:17.071889 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:17.105632 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b1338-c71f-406c-88d6-38989b0020cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:17.106264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b1338-c71f-406c-88d6-38989b0020cf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.106547 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b1338-c71f-406c-88d6-38989b0020cf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.106822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b1338-c71f-406c-88d6-38989b0020cf None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.107483 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b1338-c71f-406c-88d6-38989b0020cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:17.108879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b1338-c71f-406c-88d6-38989b0020cf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:17.115990 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 373/754] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:15:17.692724 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f881b9-5b0c-4cb1-bc62-5f7749c7e83e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.693232 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f881b9-5b0c-4cb1-bc62-5f7749c7e83e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.693309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f881b9-5b0c-4cb1-bc62-5f7749c7e83e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.694898 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 382/755] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:17.700107 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dcb94-2b1a-4aed-ab87-28c17b2060f1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.700524 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dcb94-2b1a-4aed-ab87-28c17b2060f1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.700732 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dcb94-2b1a-4aed-ab87-28c17b2060f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.719479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-309dcb94-2b1a-4aed-ab87-28c17b2060f1 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:17.721306 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-309dcb94-2b1a-4aed-ab87-28c17b2060f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:17.792005 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 374/756] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:17 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:18.063811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f881b9-5b0c-4cb1-bc62-5f7749c7e83e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.109478 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.145708 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875d51f-9a21-4d39-8293-fb20fc657bf1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.146048 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875d51f-9a21-4d39-8293-fb20fc657bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.146316 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875d51f-9a21-4d39-8293-fb20fc657bf1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.147933 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 375/757] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:18.153547 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea3e8a-141d-411e-aa98-71f6524dd7d3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.153963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea3e8a-141d-411e-aa98-71f6524dd7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.154400 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea3e8a-141d-411e-aa98-71f6524dd7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.172814 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-53ea3e8a-141d-411e-aa98-71f6524dd7d3 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:18.174687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-53ea3e8a-141d-411e-aa98-71f6524dd7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:18.176303 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f881b9-5b0c-4cb1-bc62-5f7749c7e83e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.215381 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ea669-9ca1-4539-8998-f83ae2c1e117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.216226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ea669-9ca1-4539-8998-f83ae2c1e117 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.216673 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ea669-9ca1-4539-8998-f83ae2c1e117 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.217110 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ea669-9ca1-4539-8998-f83ae2c1e117 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.217882 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ea669-9ca1-4539-8998-f83ae2c1e117 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.219547 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ea669-9ca1-4539-8998-f83ae2c1e117 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.238202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.sql.core [None req-ab2ea669-9ca1-4539-8998-f83ae2c1e117 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:15:18.238202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 07:15:18.238202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [parameters: {'user_id': '0f49a490964d4cb6b8ed3659fdd30c95', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 07:15:18.238202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81400) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ab2ea669-9ca1-4539-8998-f83ae2c1e117 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Nov 02 07:15:18.248979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:15:18.249873 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application yield resource Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise self.value Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application session.commit() Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:15:18.250885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:15:18.251805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = connection._execute_20( Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:15:18.252739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:15:18.253558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [parameters: {'user_id': '0f49a490964d4cb6b8ed3659fdd30c95', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:18.254371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Nov 02 07:15:18.255199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Nov 02 07:15:18.256098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Nov 02 07:15:18.256098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Nov 02 07:15:18.256098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:15:18.256098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:15:18.256098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 07:15:18.256098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:18.258055 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 383/759] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Nov 2 07:15:18 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:18.258265 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 376/759] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:18 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:18.264699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ea669-9ca1-4539-8998-f83ae2c1e117 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.371453 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ea669-9ca1-4539-8998-f83ae2c1e117 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.373356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.409597 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52e103-489e-4c11-93d6-8141c16f2989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.410524 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52e103-489e-4c11-93d6-8141c16f2989 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.411022 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52e103-489e-4c11-93d6-8141c16f2989 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.411493 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52e103-489e-4c11-93d6-8141c16f2989 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.419227 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52e103-489e-4c11-93d6-8141c16f2989 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.421056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52e103-489e-4c11-93d6-8141c16f2989 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8c52e103-489e-4c11-93d6-8141c16f2989 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:18.433189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:18.434151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:18.435078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:18.435078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:18.435078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:18.435078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:18.435078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:18.435078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:18.435078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:18.435078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:18.435078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:15:18.435078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:18.437255 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 384/760] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:18.445633 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea3e8a-141d-411e-aa98-71f6524dd7d3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.477999 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.555764 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52e103-489e-4c11-93d6-8141c16f2989 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.562781 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea3e8a-141d-411e-aa98-71f6524dd7d3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.607402 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4929347-f088-428b-9124-87bab55fe7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.608017 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4929347-f088-428b-9124-87bab55fe7be None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.608295 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4929347-f088-428b-9124-87bab55fe7be None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.608579 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4929347-f088-428b-9124-87bab55fe7be None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.609721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4929347-f088-428b-9124-87bab55fe7be None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:18.610101 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4929347-f088-428b-9124-87bab55fe7be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.611586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4929347-f088-428b-9124-87bab55fe7be None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.652446 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 377/761] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:18.653733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9e858-1dcb-46cb-a593-e81025ecbaa0 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.654088 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9e858-1dcb-46cb-a593-e81025ecbaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.654426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9e858-1dcb-46cb-a593-e81025ecbaa0 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.660478 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 378/762] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:18.661954 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffd84b-4b0d-440f-afbd-3e35ef01c6e9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.662246 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffd84b-4b0d-440f-afbd-3e35ef01c6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.662524 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffd84b-4b0d-440f-afbd-3e35ef01c6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.675849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52e103-489e-4c11-93d6-8141c16f2989 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.682347 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-38ffd84b-4b0d-440f-afbd-3e35ef01c6e9 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:18.683682 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-38ffd84b-4b0d-440f-afbd-3e35ef01c6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:18.716461 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4138b4-e53e-4837-a0f2-effc98007bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.717962 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4138b4-e53e-4837-a0f2-effc98007bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.718446 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4138b4-e53e-4837-a0f2-effc98007bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.718998 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4138b4-e53e-4837-a0f2-effc98007bc8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.719825 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4138b4-e53e-4837-a0f2-effc98007bc8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.722122 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4138b4-e53e-4837-a0f2-effc98007bc8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9c4138b4-e53e-4837-a0f2-effc98007bc8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:18.732464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:18.733761 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:15:18.734688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:18.737143 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 385/763] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:18.743132 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4138b4-e53e-4837-a0f2-effc98007bc8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.779362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.785253 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 379/764] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:18 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:18.866534 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4138b4-e53e-4837-a0f2-effc98007bc8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.905843 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d14b40-6afd-46c8-a797-eaf6ccd75bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.906392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14b40-6afd-46c8-a797-eaf6ccd75bec None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.906677 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14b40-6afd-46c8-a797-eaf6ccd75bec None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.906951 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14b40-6afd-46c8-a797-eaf6ccd75bec None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.907649 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d14b40-6afd-46c8-a797-eaf6ccd75bec None admin] RBAC: Adding query filter params (name=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:18.907954 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d14b40-6afd-46c8-a797-eaf6ccd75bec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.909342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d14b40-6afd-46c8-a797-eaf6ccd75bec None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.917799 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 386/765] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:18.923999 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffd84b-4b0d-440f-afbd-3e35ef01c6e9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.943979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:19.030439 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffd84b-4b0d-440f-afbd-3e35ef01c6e9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.063965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ad00f-94de-475c-9f92-76b3e8adde6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.064599 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ad00f-94de-475c-9f92-76b3e8adde6e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.064881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ad00f-94de-475c-9f92-76b3e8adde6e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.065150 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ad00f-94de-475c-9f92-76b3e8adde6e None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.065771 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ad00f-94de-475c-9f92-76b3e8adde6e None admin] RBAC: Adding query filter params (type=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:19.066075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ad00f-94de-475c-9f92-76b3e8adde6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.067473 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ad00f-94de-475c-9f92-76b3e8adde6e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.077194 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 380/766] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:19.077353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d14b40-6afd-46c8-a797-eaf6ccd75bec None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.078035 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ad00f-94de-475c-9f92-76b3e8adde6e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.163498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:19.172071 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:19.182625 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d14b40-6afd-46c8-a797-eaf6ccd75bec None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.185130 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ad00f-94de-475c-9f92-76b3e8adde6e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.218904 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e07355-fa43-4145-a0f8-da9168ae5205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.219545 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e07355-fa43-4145-a0f8-da9168ae5205 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.219822 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e07355-fa43-4145-a0f8-da9168ae5205 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.220707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e07355-fa43-4145-a0f8-da9168ae5205 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.221297 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e07355-fa43-4145-a0f8-da9168ae5205 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.222667 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e07355-fa43-4145-a0f8-da9168ae5205 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.223112 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe76ac2-93e3-41fd-94be-8d65ce035fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.224096 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe76ac2-93e3-41fd-94be-8d65ce035fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.224594 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe76ac2-93e3-41fd-94be-8d65ce035fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.225051 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe76ac2-93e3-41fd-94be-8d65ce035fc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.226092 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe76ac2-93e3-41fd-94be-8d65ce035fc0 None admin] RBAC: Adding query filter params (interface=public, service_id=2d6be3ffaf40490bb9c5d4b6dd638007, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:19.226406 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe76ac2-93e3-41fd-94be-8d65ce035fc0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.227879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe76ac2-93e3-41fd-94be-8d65ce035fc0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-69e07355-fa43-4145-a0f8-da9168ae5205 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:19.233783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:19.234838 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:19.241747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 387/767] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:19.242526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 381/768] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Nov 2 07:15:19 2022] GET /identity/v3/endpoints?service_id=2d6be3ffaf40490bb9c5d4b6dd638007&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:19.242843 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e07355-fa43-4145-a0f8-da9168ae5205 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.244031 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe76ac2-93e3-41fd-94be-8d65ce035fc0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.350661 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe76ac2-93e3-41fd-94be-8d65ce035fc0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.357415 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e07355-fa43-4145-a0f8-da9168ae5205 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.384963 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058ff60-a51d-4bc9-bca1-787b7a7b21f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.385565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058ff60-a51d-4bc9-bca1-787b7a7b21f0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.385842 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058ff60-a51d-4bc9-bca1-787b7a7b21f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.386109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058ff60-a51d-4bc9-bca1-787b7a7b21f0 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.386728 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058ff60-a51d-4bc9-bca1-787b7a7b21f0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:19.387030 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058ff60-a51d-4bc9-bca1-787b7a7b21f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.388398 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058ff60-a51d-4bc9-bca1-787b7a7b21f0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.391813 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afad1d-0b16-4d4c-b6a2-ca9818ef5565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.392409 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afad1d-0b16-4d4c-b6a2-ca9818ef5565 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.392721 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afad1d-0b16-4d4c-b6a2-ca9818ef5565 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.392992 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afad1d-0b16-4d4c-b6a2-ca9818ef5565 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.397708 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 382/769] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:19.397847 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afad1d-0b16-4d4c-b6a2-ca9818ef5565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.397925 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86785f7e-8eb8-47f2-85d3-7bb7dd1e7490 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.397925 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86785f7e-8eb8-47f2-85d3-7bb7dd1e7490 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.397925 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86785f7e-8eb8-47f2-85d3-7bb7dd1e7490 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.398173 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afad1d-0b16-4d4c-b6a2-ca9818ef5565 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.398258 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 383/770] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:19.400943 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86785f7e-8eb8-47f2-85d3-7bb7dd1e7490 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.402084 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 388/771] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Nov 2 07:15:19 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:15:19.403333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ce3d2-860d-45b9-be80-f2cc0c1a7d1e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.403811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ce3d2-860d-45b9-be80-f2cc0c1a7d1e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.404086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ce3d2-860d-45b9-be80-f2cc0c1a7d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.413062 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:19.426593 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3a4ce3d2-860d-45b9-be80-f2cc0c1a7d1e None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:19.428113 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3a4ce3d2-860d-45b9-be80-f2cc0c1a7d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:19.444356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:19.511171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 389/772] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:19 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:19.533287 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86785f7e-8eb8-47f2-85d3-7bb7dd1e7490 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.571098 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39accff-45e5-48bd-832d-de970a2355a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.571711 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39accff-45e5-48bd-832d-de970a2355a9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.571988 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39accff-45e5-48bd-832d-de970a2355a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.572252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39accff-45e5-48bd-832d-de970a2355a9 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.572905 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39accff-45e5-48bd-832d-de970a2355a9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:19.573203 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39accff-45e5-48bd-832d-de970a2355a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.574590 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39accff-45e5-48bd-832d-de970a2355a9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.586572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 384/773] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:19.588202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ce3d2-860d-45b9-be80-f2cc0c1a7d1e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.698614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ce3d2-860d-45b9-be80-f2cc0c1a7d1e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.733342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fc632-bdfc-42a2-b8cd-42e3afb094de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.733990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fc632-bdfc-42a2-b8cd-42e3afb094de None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.734209 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fc632-bdfc-42a2-b8cd-42e3afb094de None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.734473 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fc632-bdfc-42a2-b8cd-42e3afb094de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.735129 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972fc632-bdfc-42a2-b8cd-42e3afb094de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.737081 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972fc632-bdfc-42a2-b8cd-42e3afb094de None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.757519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 390/774] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Nov 2 07:15:19 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:19.785230 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39accff-45e5-48bd-832d-de970a2355a9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.867201 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:19.901111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39accff-45e5-48bd-832d-de970a2355a9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.944615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a522e-6233-4f50-b5e6-af3fdb4b4006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.945264 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a522e-6233-4f50-b5e6-af3fdb4b4006 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.945544 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a522e-6233-4f50-b5e6-af3fdb4b4006 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.945812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a522e-6233-4f50-b5e6-af3fdb4b4006 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.946386 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a522e-6233-4f50-b5e6-af3fdb4b4006 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.947738 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a522e-6233-4f50-b5e6-af3fdb4b4006 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7b8a522e-6233-4f50-b5e6-af3fdb4b4006 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:19.956674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:19.957874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:19.958800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 385/775] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Nov 2 07:15:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:19.964327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fc632-bdfc-42a2-b8cd-42e3afb094de None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.968191 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:20.072996 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fc632-bdfc-42a2-b8cd-42e3afb094de None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:20.107028 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2c7bc-5ef9-4909-b759-89687e574273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:20.107644 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2c7bc-5ef9-4909-b759-89687e574273 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.107921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2c7bc-5ef9-4909-b759-89687e574273 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.108191 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2c7bc-5ef9-4909-b759-89687e574273 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.108846 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2c7bc-5ef9-4909-b759-89687e574273 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:20.109151 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2c7bc-5ef9-4909-b759-89687e574273 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:20.110492 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2c7bc-5ef9-4909-b759-89687e574273 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:20.120217 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 391/776] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:20.121331 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a522e-6233-4f50-b5e6-af3fdb4b4006 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:20.223622 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a522e-6233-4f50-b5e6-af3fdb4b4006 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:20.252750 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:20.259094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a766221-f716-44c3-9cf7-ac3df58163df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:20.259706 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a766221-f716-44c3-9cf7-ac3df58163df None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.259985 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a766221-f716-44c3-9cf7-ac3df58163df None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.260257 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a766221-f716-44c3-9cf7-ac3df58163df None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.260909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a766221-f716-44c3-9cf7-ac3df58163df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:20.261213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a766221-f716-44c3-9cf7-ac3df58163df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:20.262574 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a766221-f716-44c3-9cf7-ac3df58163df None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:20.272895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 386/777] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:20.274228 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2c7bc-5ef9-4909-b759-89687e574273 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:20.348288 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:20.385593 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2c7bc-5ef9-4909-b759-89687e574273 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:20.418442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e76fe8-e14b-4c96-b281-880dc96a6ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:20.419049 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e76fe8-e14b-4c96-b281-880dc96a6ab9 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.419328 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e76fe8-e14b-4c96-b281-880dc96a6ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.419600 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e76fe8-e14b-4c96-b281-880dc96a6ab9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.420261 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e76fe8-e14b-4c96-b281-880dc96a6ab9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:20.421684 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e76fe8-e14b-4c96-b281-880dc96a6ab9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:20.439040 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 392/778] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Nov 2 07:15:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:21.590904 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2ff5d-07dd-41a2-b2a6-4b08e7b30a22 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.591458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2ff5d-07dd-41a2-b2a6-4b08e7b30a22 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.591458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2ff5d-07dd-41a2-b2a6-4b08e7b30a22 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.592864 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 387/779] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:21.598519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a023eb-0900-4b61-a5b8-83df11b66137 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.598910 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a023eb-0900-4b61-a5b8-83df11b66137 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.599089 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a023eb-0900-4b61-a5b8-83df11b66137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.617329 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-39a023eb-0900-4b61-a5b8-83df11b66137 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:21.618519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-39a023eb-0900-4b61-a5b8-83df11b66137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:21.699945 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 393/780] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:21.969798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2ff5d-07dd-41a2-b2a6-4b08e7b30a22 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.999465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a7532-0820-4b94-98cf-3211d9b8d07b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.999797 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a7532-0820-4b94-98cf-3211d9b8d07b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.000022 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a7532-0820-4b94-98cf-3211d9b8d07b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.001488 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 394/781] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:22.006298 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e74274-fa56-42c7-ac06-442f8c594990 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.006592 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e74274-fa56-42c7-ac06-442f8c594990 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.006849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e74274-fa56-42c7-ac06-442f8c594990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.025576 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-94e74274-fa56-42c7-ac06-442f8c594990 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:22.026753 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-94e74274-fa56-42c7-ac06-442f8c594990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:22.079587 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.081491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2ff5d-07dd-41a2-b2a6-4b08e7b30a22 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.098714 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 395/782] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:22.121127 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaabe5c-3c93-468f-8895-c1de7206a2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.121759 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaabe5c-3c93-468f-8895-c1de7206a2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.122039 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaabe5c-3c93-468f-8895-c1de7206a2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.122313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaabe5c-3c93-468f-8895-c1de7206a2e8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.122887 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaabe5c-3c93-468f-8895-c1de7206a2e8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.124239 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaabe5c-3c93-468f-8895-c1de7206a2e8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7eaabe5c-3c93-468f-8895-c1de7206a2e8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:22.133222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:22.134282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:22.135111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 388/783] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:22.140277 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e74274-fa56-42c7-ac06-442f8c594990 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.238775 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e74274-fa56-42c7-ac06-442f8c594990 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.272850 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d7dcd-49b3-4760-bdd4-2d8970695124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.273458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d7dcd-49b3-4760-bdd4-2d8970695124 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.273731 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d7dcd-49b3-4760-bdd4-2d8970695124 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.273999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d7dcd-49b3-4760-bdd4-2d8970695124 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.274782 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d7dcd-49b3-4760-bdd4-2d8970695124 None admin] RBAC: Adding query filter params (name=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:22.275087 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d7dcd-49b3-4760-bdd4-2d8970695124 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.276424 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d7dcd-49b3-4760-bdd4-2d8970695124 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.286266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 396/784] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:22.287766 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaabe5c-3c93-468f-8895-c1de7206a2e8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.348380 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.377395 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d7dcd-49b3-4760-bdd4-2d8970695124 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.392642 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaabe5c-3c93-468f-8895-c1de7206a2e8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.429699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8b052-1323-4c7e-802e-9a237cbf686a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.430316 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8b052-1323-4c7e-802e-9a237cbf686a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.430600 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8b052-1323-4c7e-802e-9a237cbf686a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.430876 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8b052-1323-4c7e-802e-9a237cbf686a None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.431550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8b052-1323-4c7e-802e-9a237cbf686a None admin] RBAC: Adding query filter params (type=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:22.431827 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8b052-1323-4c7e-802e-9a237cbf686a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.433328 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8b052-1323-4c7e-802e-9a237cbf686a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.435632 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.438957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 389/785] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:22.444573 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8b052-1323-4c7e-802e-9a237cbf686a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.494903 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d7dcd-49b3-4760-bdd4-2d8970695124 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.537731 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32737402-b1b7-4958-a38f-f012ba585863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.538743 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32737402-b1b7-4958-a38f-f012ba585863 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.539223 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32737402-b1b7-4958-a38f-f012ba585863 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.539702 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32737402-b1b7-4958-a38f-f012ba585863 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.550510 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8b052-1323-4c7e-802e-9a237cbf686a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.553511 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32737402-b1b7-4958-a38f-f012ba585863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-32737402-b1b7-4958-a38f-f012ba585863 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:22.558933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:22.559862 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:15:22.560854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:22.562495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 397/786] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:22.568484 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32737402-b1b7-4958-a38f-f012ba585863 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.575815 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.589158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693c20d-6acf-4dc7-bfdc-7cb844f58093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.589765 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693c20d-6acf-4dc7-bfdc-7cb844f58093 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.590041 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693c20d-6acf-4dc7-bfdc-7cb844f58093 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.590308 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693c20d-6acf-4dc7-bfdc-7cb844f58093 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.590964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693c20d-6acf-4dc7-bfdc-7cb844f58093 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.592322 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693c20d-6acf-4dc7-bfdc-7cb844f58093 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.624570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 390/787] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Nov 2 07:15:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:22.624570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab64d21-6cf6-4d17-ac19-e09402690e18 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.624570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab64d21-6cf6-4d17-ac19-e09402690e18 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.624570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab64d21-6cf6-4d17-ac19-e09402690e18 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.624570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 391/788] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:22.624570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481241e4-e3b7-43d1-8349-7361b8017666 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.624570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481241e4-e3b7-43d1-8349-7361b8017666 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.625190 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481241e4-e3b7-43d1-8349-7361b8017666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.641471 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-481241e4-e3b7-43d1-8349-7361b8017666 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:22.643252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-481241e4-e3b7-43d1-8349-7361b8017666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:22.699965 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32737402-b1b7-4958-a38f-f012ba585863 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.711308 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.732314 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 392/789] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:22.740116 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1018c9d-480c-478c-aa51-0dcfc4df3e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.740779 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1018c9d-480c-478c-aa51-0dcfc4df3e37 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.741061 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1018c9d-480c-478c-aa51-0dcfc4df3e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.741334 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1018c9d-480c-478c-aa51-0dcfc4df3e37 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.742021 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1018c9d-480c-478c-aa51-0dcfc4df3e37 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:22.742323 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1018c9d-480c-478c-aa51-0dcfc4df3e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.743724 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1018c9d-480c-478c-aa51-0dcfc4df3e37 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.754143 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 398/790] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:22.759884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481241e4-e3b7-43d1-8349-7361b8017666 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.867538 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481241e4-e3b7-43d1-8349-7361b8017666 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.874866 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.904790 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b71411-4fba-4b87-ac53-362ef895ffe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.905422 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b71411-4fba-4b87-ac53-362ef895ffe7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.905697 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b71411-4fba-4b87-ac53-362ef895ffe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.905969 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b71411-4fba-4b87-ac53-362ef895ffe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.916695 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b71411-4fba-4b87-ac53-362ef895ffe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.918138 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b71411-4fba-4b87-ac53-362ef895ffe7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-36b71411-4fba-4b87-ac53-362ef895ffe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:22.932644 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:22.933746 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:22.934619 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:22.934619 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:22.934619 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:22.934619 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 393/791] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:22.940648 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1018c9d-480c-478c-aa51-0dcfc4df3e37 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.005342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b71411-4fba-4b87-ac53-362ef895ffe7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.059031 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1018c9d-480c-478c-aa51-0dcfc4df3e37 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.094993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8163f6-c5bb-4a88-ad60-b39c96030f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.095606 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8163f6-c5bb-4a88-ad60-b39c96030f94 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.095892 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8163f6-c5bb-4a88-ad60-b39c96030f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.096162 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8163f6-c5bb-4a88-ad60-b39c96030f94 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.096832 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8163f6-c5bb-4a88-ad60-b39c96030f94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:23.097133 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8163f6-c5bb-4a88-ad60-b39c96030f94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.098496 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8163f6-c5bb-4a88-ad60-b39c96030f94 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.110583 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 399/792] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:23.113207 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b71411-4fba-4b87-ac53-362ef895ffe7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.116119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8163f6-c5bb-4a88-ad60-b39c96030f94 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.149007 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.154818 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee31bc0-c532-47cc-8745-51c98ffc9259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.155692 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31bc0-c532-47cc-8745-51c98ffc9259 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.156153 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31bc0-c532-47cc-8745-51c98ffc9259 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.156646 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31bc0-c532-47cc-8745-51c98ffc9259 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.158480 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee31bc0-c532-47cc-8745-51c98ffc9259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.160097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee31bc0-c532-47cc-8745-51c98ffc9259 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.162462 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 394/793] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:23.167859 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee31bc0-c532-47cc-8745-51c98ffc9259 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.197359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.227480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8163f6-c5bb-4a88-ad60-b39c96030f94 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.264876 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0e6a1-dce0-4a13-84dd-85112e2da65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.266315 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0e6a1-dce0-4a13-84dd-85112e2da65d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.267402 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0e6a1-dce0-4a13-84dd-85112e2da65d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.267851 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0e6a1-dce0-4a13-84dd-85112e2da65d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.274965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee31bc0-c532-47cc-8745-51c98ffc9259 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.275247 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0e6a1-dce0-4a13-84dd-85112e2da65d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.276723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0e6a1-dce0-4a13-84dd-85112e2da65d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7bb0e6a1-dce0-4a13-84dd-85112e2da65d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:23.288612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:23.289639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:23.291357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:23.291357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:23.291357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:23.291357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:23.291357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:23.291357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:23.291357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:23.291357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:23.291357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:23.291357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:23.291357 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 400/794] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:23.295832 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0e6a1-dce0-4a13-84dd-85112e2da65d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.313243 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c6bc8-4bf5-47eb-a5ad-3ba5b73bc916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.313966 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6bc8-4bf5-47eb-a5ad-3ba5b73bc916 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.314241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6bc8-4bf5-47eb-a5ad-3ba5b73bc916 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.314418 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6bc8-4bf5-47eb-a5ad-3ba5b73bc916 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.315063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c6bc8-4bf5-47eb-a5ad-3ba5b73bc916 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.316498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c6bc8-4bf5-47eb-a5ad-3ba5b73bc916 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.346137 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 395/795] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:15:23 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:23.400915 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.413202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0e6a1-dce0-4a13-84dd-85112e2da65d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.452853 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acc57c-d3e7-4dec-8a09-9a57be78c973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.453533 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acc57c-d3e7-4dec-8a09-9a57be78c973 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.453921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acc57c-d3e7-4dec-8a09-9a57be78c973 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.454243 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acc57c-d3e7-4dec-8a09-9a57be78c973 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.455010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acc57c-d3e7-4dec-8a09-9a57be78c973 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:23.455318 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acc57c-d3e7-4dec-8a09-9a57be78c973 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.456871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acc57c-d3e7-4dec-8a09-9a57be78c973 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.477689 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 401/796] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:23.483585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c6bc8-4bf5-47eb-a5ad-3ba5b73bc916 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.574296 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.598501 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c6bc8-4bf5-47eb-a5ad-3ba5b73bc916 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.634461 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a071eaf-2173-491a-8ef9-6331d64115dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.635093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a071eaf-2173-491a-8ef9-6331d64115dc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.635370 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a071eaf-2173-491a-8ef9-6331d64115dc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.635652 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a071eaf-2173-491a-8ef9-6331d64115dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.647947 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a071eaf-2173-491a-8ef9-6331d64115dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.649432 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a071eaf-2173-491a-8ef9-6331d64115dc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1a071eaf-2173-491a-8ef9-6331d64115dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:23.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:23.665737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:23.666639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:23.666639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:23.666639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:23.666639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 396/797] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:23.671637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acc57c-d3e7-4dec-8a09-9a57be78c973 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.773170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acc57c-d3e7-4dec-8a09-9a57be78c973 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.807473 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce514a3-6efb-408c-ae2a-93ef7dc5d7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.808081 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce514a3-6efb-408c-ae2a-93ef7dc5d7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.808360 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce514a3-6efb-408c-ae2a-93ef7dc5d7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.808659 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce514a3-6efb-408c-ae2a-93ef7dc5d7d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.809298 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce514a3-6efb-408c-ae2a-93ef7dc5d7d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:23.809602 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce514a3-6efb-408c-ae2a-93ef7dc5d7d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.810956 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce514a3-6efb-408c-ae2a-93ef7dc5d7d0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.822483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 402/798] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:23.827897 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a071eaf-2173-491a-8ef9-6331d64115dc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:23.930313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a071eaf-2173-491a-8ef9-6331d64115dc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:23.958105 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:23.965107 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b25ce3-7a54-4ba8-ba20-5ec6cc783b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:23.965738 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b25ce3-7a54-4ba8-ba20-5ec6cc783b2a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.966015 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b25ce3-7a54-4ba8-ba20-5ec6cc783b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.966284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b25ce3-7a54-4ba8-ba20-5ec6cc783b2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.973538 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b25ce3-7a54-4ba8-ba20-5ec6cc783b2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:23.974961 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b25ce3-7a54-4ba8-ba20-5ec6cc783b2a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-47b25ce3-7a54-4ba8-ba20-5ec6cc783b2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:23.986103 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:23.986995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:23.987881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:23.987881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:23.987881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:23.987881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:23.987881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:23.987881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:23.987881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:23.987881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:23.987881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 397/799] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:23.996660 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce514a3-6efb-408c-ae2a-93ef7dc5d7d0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:24.008272 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:24.100747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce514a3-6efb-408c-ae2a-93ef7dc5d7d0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.133780 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbae6b2-c500-4acf-862e-46d4d0abeb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.134393 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbae6b2-c500-4acf-862e-46d4d0abeb78 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.134672 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbae6b2-c500-4acf-862e-46d4d0abeb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.134944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbae6b2-c500-4acf-862e-46d4d0abeb78 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.135631 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbae6b2-c500-4acf-862e-46d4d0abeb78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:24.135943 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbae6b2-c500-4acf-862e-46d4d0abeb78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:24.137401 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbae6b2-c500-4acf-862e-46d4d0abeb78 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:24.150016 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 403/800] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:24.155699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b25ce3-7a54-4ba8-ba20-5ec6cc783b2a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:24.256113 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b25ce3-7a54-4ba8-ba20-5ec6cc783b2a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.292837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1addd02-f8c2-4328-afb7-09ccfda6e1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.293571 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1addd02-f8c2-4328-afb7-09ccfda6e1bf None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.293844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1addd02-f8c2-4328-afb7-09ccfda6e1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.294146 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1addd02-f8c2-4328-afb7-09ccfda6e1bf None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/00ab674b018e474ab363e8e17d225e27/roles/d660bf006ef649159f7369c7f1623c04` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.296289 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1addd02-f8c2-4328-afb7-09ccfda6e1bf None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=00ab674b018e474ab363e8e17d225e27, role_id=d660bf006ef649159f7369c7f1623c04)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:24.297800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1addd02-f8c2-4328-afb7-09ccfda6e1bf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:24.312237 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 398/801] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:24 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/00ab674b018e474ab363e8e17d225e27/roles/d660bf006ef649159f7369c7f1623c04 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:24.845981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f80406-9fa2-44c1-9628-e32997f25dbc None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.846493 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f80406-9fa2-44c1-9628-e32997f25dbc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.846570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f80406-9fa2-44c1-9628-e32997f25dbc None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.847985 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 404/802] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:24.853184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd4f2b-e480-474a-90b1-0bd59b7550d7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.853549 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd4f2b-e480-474a-90b1-0bd59b7550d7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.853743 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd4f2b-e480-474a-90b1-0bd59b7550d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.871970 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-50dd4f2b-e480-474a-90b1-0bd59b7550d7 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:24.873190 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-50dd4f2b-e480-474a-90b1-0bd59b7550d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:24.919023 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:24.942050 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 399/803] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:25.210588 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f80406-9fa2-44c1-9628-e32997f25dbc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.281326 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:25.317993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f80406-9fa2-44c1-9628-e32997f25dbc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.353230 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b26e03-79d4-4ae8-8239-ff693228608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.353827 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b26e03-79d4-4ae8-8239-ff693228608a None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.354101 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b26e03-79d4-4ae8-8239-ff693228608a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.354368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b26e03-79d4-4ae8-8239-ff693228608a None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.355032 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b26e03-79d4-4ae8-8239-ff693228608a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:25.356395 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b26e03-79d4-4ae8-8239-ff693228608a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:25.364796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.sql.core [None req-41b26e03-79d4-4ae8-8239-ff693228608a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:15:25.364796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:15:25.364796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [parameters: {'id': '4c264fb8f1dc41f59188ce61703ff826', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:15:25.364796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81400) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-41b26e03-79d4-4ae8-8239-ff693228608a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:15:25.373837 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:15:25.374638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application yield resource Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise self.value Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application session.commit() Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:15:25.377214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:15:25.378093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:15:25.378975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:15:25.379750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [parameters: {'id': '4c264fb8f1dc41f59188ce61703ff826', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:15:25.380642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:25.381777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:25.383878 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 405/804] 10.222.0.15 () {60 vars in 1137 bytes} [Wed Nov 2 07:15:25 2022] POST /identity/v3/domains => generated 190 bytes in 165 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:25.385542 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd4f2b-e480-474a-90b1-0bd59b7550d7 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.494029 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd4f2b-e480-474a-90b1-0bd59b7550d7 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.528990 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90764d27-c48c-40d1-9276-5695567ce704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.529622 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90764d27-c48c-40d1-9276-5695567ce704 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.529898 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90764d27-c48c-40d1-9276-5695567ce704 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.530166 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90764d27-c48c-40d1-9276-5695567ce704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.541177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90764d27-c48c-40d1-9276-5695567ce704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:25.542617 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90764d27-c48c-40d1-9276-5695567ce704 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:25.546332 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7dadf-b94c-4631-8814-635eadab0d50 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.546614 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7dadf-b94c-4631-8814-635eadab0d50 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.546884 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7dadf-b94c-4631-8814-635eadab0d50 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.548378 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 406/805] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:25.553078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7048-2c55-425c-8668-dd1115870e82 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.553358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7048-2c55-425c-8668-dd1115870e82 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.553626 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7048-2c55-425c-8668-dd1115870e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-90764d27-c48c-40d1-9276-5695567ce704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:25.557491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:25.558495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:25.559476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:25.559476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:25.559476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:25.559476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 400/806] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:25.564263 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90764d27-c48c-40d1-9276-5695567ce704 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.574292 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c42e7048-2c55-425c-8668-dd1115870e82 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:25.575473 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c42e7048-2c55-425c-8668-dd1115870e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:25.602845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:25.649054 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 407/807] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:25.684569 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90764d27-c48c-40d1-9276-5695567ce704 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.719193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a67e8f-4f30-4884-8dff-6a882bfe806b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.720605 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a67e8f-4f30-4884-8dff-6a882bfe806b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.721046 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a67e8f-4f30-4884-8dff-6a882bfe806b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.721474 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a67e8f-4f30-4884-8dff-6a882bfe806b None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.722260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a67e8f-4f30-4884-8dff-6a882bfe806b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:25.722723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a67e8f-4f30-4884-8dff-6a882bfe806b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:25.724222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a67e8f-4f30-4884-8dff-6a882bfe806b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:25.736454 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 401/808] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:25.916593 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e7048-2c55-425c-8668-dd1115870e82 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.986104 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:26.030778 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e7048-2c55-425c-8668-dd1115870e82 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:26.068661 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d579e-46f7-48ac-bb11-2846d081ce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:26.069306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d579e-46f7-48ac-bb11-2846d081ce2b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.069586 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d579e-46f7-48ac-bb11-2846d081ce2b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.069858 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d579e-46f7-48ac-bb11-2846d081ce2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.070967 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d579e-46f7-48ac-bb11-2846d081ce2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:26.072396 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d579e-46f7-48ac-bb11-2846d081ce2b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:26.074018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 408/809] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:26.079812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a67e8f-4f30-4884-8dff-6a882bfe806b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:26.169303 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:26.178495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a67e8f-4f30-4884-8dff-6a882bfe806b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:26.210937 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19904c6f-a1bc-463c-80ed-98e6c883c3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:26.211529 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19904c6f-a1bc-463c-80ed-98e6c883c3d1 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.212057 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19904c6f-a1bc-463c-80ed-98e6c883c3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.212320 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19904c6f-a1bc-463c-80ed-98e6c883c3d1 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.212995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19904c6f-a1bc-463c-80ed-98e6c883c3d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:26.214404 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19904c6f-a1bc-463c-80ed-98e6c883c3d1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:26.235155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 402/810] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:26 2022] POST /identity/v3/users => generated 285 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:26.465389 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff6cfb-27ff-4aa2-9539-88d8f1e4d140 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.465579 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff6cfb-27ff-4aa2-9539-88d8f1e4d140 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.465853 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff6cfb-27ff-4aa2-9539-88d8f1e4d140 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.467336 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 409/811] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:26.472481 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0531b-1fb3-4957-91ae-66bf87987e04 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.472806 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0531b-1fb3-4957-91ae-66bf87987e04 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.473084 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0531b-1fb3-4957-91ae-66bf87987e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.491374 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6ed0531b-1fb3-4957-91ae-66bf87987e04 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:26.492570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6ed0531b-1fb3-4957-91ae-66bf87987e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:26.557277 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 403/812] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:26.824156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff6cfb-27ff-4aa2-9539-88d8f1e4d140 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:26.922897 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff6cfb-27ff-4aa2-9539-88d8f1e4d140 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:26.950573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:26.957594 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4761ca-99bc-487b-aab7-35a5c8917b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:26.958228 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4761ca-99bc-487b-aab7-35a5c8917b71 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.958507 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4761ca-99bc-487b-aab7-35a5c8917b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.958778 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4761ca-99bc-487b-aab7-35a5c8917b71 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.969080 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4761ca-99bc-487b-aab7-35a5c8917b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-de4761ca-99bc-487b-aab7-35a5c8917b71 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:26.973920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:26.974938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:15:26.975849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:26.980079 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 410/813] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Nov 2 07:15:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:26.981155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0531b-1fb3-4957-91ae-66bf87987e04 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:27.058622 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:27.080722 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0531b-1fb3-4957-91ae-66bf87987e04 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:27.113614 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3610ad1-fc47-45f6-adb3-de1dfa01ef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:27.114223 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3610ad1-fc47-45f6-adb3-de1dfa01ef8e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.114501 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3610ad1-fc47-45f6-adb3-de1dfa01ef8e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.114771 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3610ad1-fc47-45f6-adb3-de1dfa01ef8e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.115430 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3610ad1-fc47-45f6-adb3-de1dfa01ef8e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:27.115734 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3610ad1-fc47-45f6-adb3-de1dfa01ef8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:27.117131 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3610ad1-fc47-45f6-adb3-de1dfa01ef8e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:27.125192 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 404/814] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Nov 2 07:15:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:27.132311 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4761ca-99bc-487b-aab7-35a5c8917b71 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:27.228891 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4761ca-99bc-487b-aab7-35a5c8917b71 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:27.260918 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74415d-e5b9-4bf8-a959-2ab8073ba289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:27.261538 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74415d-e5b9-4bf8-a959-2ab8073ba289 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.261816 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74415d-e5b9-4bf8-a959-2ab8073ba289 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.262086 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74415d-e5b9-4bf8-a959-2ab8073ba289 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.272257 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74415d-e5b9-4bf8-a959-2ab8073ba289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:27.273720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74415d-e5b9-4bf8-a959-2ab8073ba289 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-da74415d-e5b9-4bf8-a959-2ab8073ba289 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:27.287726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:27.288757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:27.289629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:27.289629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:27.289629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:27.292863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 411/815] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:27.295125 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3610ad1-fc47-45f6-adb3-de1dfa01ef8e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:27.393699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3610ad1-fc47-45f6-adb3-de1dfa01ef8e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:27.426351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11d82d-a73c-4ab8-a07b-c1b8c9619ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:27.426943 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11d82d-a73c-4ab8-a07b-c1b8c9619ccf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.427216 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11d82d-a73c-4ab8-a07b-c1b8c9619ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.427481 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11d82d-a73c-4ab8-a07b-c1b8c9619ccf None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.428104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11d82d-a73c-4ab8-a07b-c1b8c9619ccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:27.428400 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11d82d-a73c-4ab8-a07b-c1b8c9619ccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:27.429784 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11d82d-a73c-4ab8-a07b-c1b8c9619ccf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:27.439390 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:27.443122 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 405/816] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:27.448465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74415d-e5b9-4bf8-a959-2ab8073ba289 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:27.491884 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:27.546811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74415d-e5b9-4bf8-a959-2ab8073ba289 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:27.578678 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab5291-6a3c-48c5-b263-cbeecf17cf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:27.579330 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab5291-6a3c-48c5-b263-cbeecf17cf09 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.579606 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab5291-6a3c-48c5-b263-cbeecf17cf09 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.579875 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab5291-6a3c-48c5-b263-cbeecf17cf09 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.587187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cab5291-6a3c-48c5-b263-cbeecf17cf09 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:27.588631 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cab5291-6a3c-48c5-b263-cbeecf17cf09 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6cab5291-6a3c-48c5-b263-cbeecf17cf09 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:27.599802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:27.600767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:27.602034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:27.602034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:27.602034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:27.602034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:27.602034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:27.602034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:27.602034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:27.602034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:27.602034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:27.602034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:27.602034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 412/817] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Nov 2 07:15:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:27.606573 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11d82d-a73c-4ab8-a07b-c1b8c9619ccf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:27.708505 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11d82d-a73c-4ab8-a07b-c1b8c9619ccf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:27.740929 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd36684-84c7-4656-9018-41efc446d85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:27.741525 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd36684-84c7-4656-9018-41efc446d85a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.741803 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd36684-84c7-4656-9018-41efc446d85a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.742073 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd36684-84c7-4656-9018-41efc446d85a None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.742781 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd36684-84c7-4656-9018-41efc446d85a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:27.743081 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd36684-84c7-4656-9018-41efc446d85a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:27.744509 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd36684-84c7-4656-9018-41efc446d85a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:27.761327 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 406/818] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Nov 2 07:15:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:27.767183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab5291-6a3c-48c5-b263-cbeecf17cf09 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:27.863282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab5291-6a3c-48c5-b263-cbeecf17cf09 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:27.864790 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:27.897550 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50f257-f8f2-4c6a-ba2d-234327e7068b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:27.898148 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50f257-f8f2-4c6a-ba2d-234327e7068b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.898421 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50f257-f8f2-4c6a-ba2d-234327e7068b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.898687 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50f257-f8f2-4c6a-ba2d-234327e7068b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.964597 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50f257-f8f2-4c6a-ba2d-234327e7068b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:27.965946 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50f257-f8f2-4c6a-ba2d-234327e7068b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.059673 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 413/819] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:15:28.065981 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd36684-84c7-4656-9018-41efc446d85a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.122265 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.163212 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd36684-84c7-4656-9018-41efc446d85a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.195489 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff819483-f407-41ed-87a5-4426af758c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.196106 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff819483-f407-41ed-87a5-4426af758c61 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.196384 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff819483-f407-41ed-87a5-4426af758c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.196687 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff819483-f407-41ed-87a5-4426af758c61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.207005 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff819483-f407-41ed-87a5-4426af758c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.208426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff819483-f407-41ed-87a5-4426af758c61 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ff819483-f407-41ed-87a5-4426af758c61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:28.223351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:28.224477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:28.225676 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:28.225676 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:28.225676 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:28.228763 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 407/820] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:28.230066 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50f257-f8f2-4c6a-ba2d-234327e7068b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.232292 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.331693 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50f257-f8f2-4c6a-ba2d-234327e7068b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.363988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b643fc-2226-4fa2-a566-b302b1815a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.364616 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b643fc-2226-4fa2-a566-b302b1815a5d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.364894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b643fc-2226-4fa2-a566-b302b1815a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.365162 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b643fc-2226-4fa2-a566-b302b1815a5d None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.365789 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b643fc-2226-4fa2-a566-b302b1815a5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:28.366084 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b643fc-2226-4fa2-a566-b302b1815a5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.367432 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b643fc-2226-4fa2-a566-b302b1815a5d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.378775 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 414/821] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:28.384343 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff819483-f407-41ed-87a5-4426af758c61 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.402225 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98653-7aab-4a1d-8bc8-23b813abd660 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.402548 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98653-7aab-4a1d-8bc8-23b813abd660 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.402831 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98653-7aab-4a1d-8bc8-23b813abd660 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.404155 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 415/822] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:28.409555 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f865-c584-452f-bdc4-cd4f512e9314 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.409839 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f865-c584-452f-bdc4-cd4f512e9314 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.410116 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f865-c584-452f-bdc4-cd4f512e9314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.428263 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2ec4f865-c584-452f-bdc4-cd4f512e9314 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:28.429489 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2ec4f865-c584-452f-bdc4-cd4f512e9314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:28.484781 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.484999 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff819483-f407-41ed-87a5-4426af758c61 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.495296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 416/823] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:28.495500 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.519462 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ca065-3819-4f8e-b8ec-4d3bf42315ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.520083 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ca065-3819-4f8e-b8ec-4d3bf42315ce None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.520357 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ca065-3819-4f8e-b8ec-4d3bf42315ce None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.520658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ca065-3819-4f8e-b8ec-4d3bf42315ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.527025 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ca065-3819-4f8e-b8ec-4d3bf42315ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.528439 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ca065-3819-4f8e-b8ec-4d3bf42315ce None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-463ca065-3819-4f8e-b8ec-4d3bf42315ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:28.539846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:28.540922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:28.541822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:28.541822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:28.541822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:28.541822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:28.541822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:28.541822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:28.541822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:28.541822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:28.541822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 408/824] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:28.546756 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4f865-c584-452f-bdc4-cd4f512e9314 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.642477 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4f865-c584-452f-bdc4-cd4f512e9314 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.676554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b1e1-4506-45b9-b7d0-1e5457f54bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.677170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b1e1-4506-45b9-b7d0-1e5457f54bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.677443 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b1e1-4506-45b9-b7d0-1e5457f54bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.677709 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b1e1-4506-45b9-b7d0-1e5457f54bf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.678383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602b1e1-4506-45b9-b7d0-1e5457f54bf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:28.678679 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602b1e1-4506-45b9-b7d0-1e5457f54bf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.680100 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602b1e1-4506-45b9-b7d0-1e5457f54bf6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.691126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 417/825] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:28.697224 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ca065-3819-4f8e-b8ec-4d3bf42315ce None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.759851 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b1e1-4506-45b9-b7d0-1e5457f54bf6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.797248 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ca065-3819-4f8e-b8ec-4d3bf42315ce None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.815057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.831081 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10876ac-47ac-4dfc-bfae-7f94803b717c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.831689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10876ac-47ac-4dfc-bfae-7f94803b717c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.831969 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10876ac-47ac-4dfc-bfae-7f94803b717c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.832239 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10876ac-47ac-4dfc-bfae-7f94803b717c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.832989 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10876ac-47ac-4dfc-bfae-7f94803b717c None admin] RBAC: Adding query filter params (role.id=d660bf006ef649159f7369c7f1623c04, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=00ab674b018e474ab363e8e17d225e27) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:28.833295 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10876ac-47ac-4dfc-bfae-7f94803b717c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.834727 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10876ac-47ac-4dfc-bfae-7f94803b717c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.840824 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 409/826] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/role_assignments?user.id=00ab674b018e474ab363e8e17d225e27&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=d660bf006ef649159f7369c7f1623c04 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:28.868857 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b1e1-4506-45b9-b7d0-1e5457f54bf6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.905943 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b601f-b312-4b21-b6e3-5ceb66e91203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.906591 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b601f-b312-4b21-b6e3-5ceb66e91203 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.906872 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b601f-b312-4b21-b6e3-5ceb66e91203 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.907144 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b601f-b312-4b21-b6e3-5ceb66e91203 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.919157 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b601f-b312-4b21-b6e3-5ceb66e91203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-235b601f-b312-4b21-b6e3-5ceb66e91203 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:28.924102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:28.925135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:28.926222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 418/827] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:28.931564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10876ac-47ac-4dfc-bfae-7f94803b717c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.968304 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:29.040067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10876ac-47ac-4dfc-bfae-7f94803b717c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.073626 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef825c3d-bb17-4a75-ae95-36ef8199e82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.074234 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef825c3d-bb17-4a75-ae95-36ef8199e82e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.074513 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef825c3d-bb17-4a75-ae95-36ef8199e82e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.074783 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef825c3d-bb17-4a75-ae95-36ef8199e82e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.075436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef825c3d-bb17-4a75-ae95-36ef8199e82e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:29.075736 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef825c3d-bb17-4a75-ae95-36ef8199e82e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.077167 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef825c3d-bb17-4a75-ae95-36ef8199e82e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.086149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 410/828] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:29.092230 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b601f-b312-4b21-b6e3-5ceb66e91203 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.196096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b601f-b312-4b21-b6e3-5ceb66e91203 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.208950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:29.230412 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532adce6-3000-4ffc-8b2d-fce8e85e0994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.231052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532adce6-3000-4ffc-8b2d-fce8e85e0994 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.231329 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532adce6-3000-4ffc-8b2d-fce8e85e0994 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.231596 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532adce6-3000-4ffc-8b2d-fce8e85e0994 None admin] PATH_INFO: `/v3/users/0160f96dd8de4e5eb683bc4888c55e6b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.245802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532adce6-3000-4ffc-8b2d-fce8e85e0994 None admin] RBAC: Authorizing `identity:get_user(user_id=0160f96dd8de4e5eb683bc4888c55e6b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.247226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532adce6-3000-4ffc-8b2d-fce8e85e0994 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.248936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 419/829] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/users/0160f96dd8de4e5eb683bc4888c55e6b => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:29.254740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef825c3d-bb17-4a75-ae95-36ef8199e82e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.465164 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:29.471674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef825c3d-bb17-4a75-ae95-36ef8199e82e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.506552 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17ac5-ae7b-4560-8242-468932627dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.507197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17ac5-ae7b-4560-8242-468932627dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.507479 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17ac5-ae7b-4560-8242-468932627dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.507757 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17ac5-ae7b-4560-8242-468932627dc1 None admin] PATH_INFO: `/v3/projects/1455a445f8e840e5ad8dcb7435d5871a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.508927 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc17ac5-ae7b-4560-8242-468932627dc1 None admin] RBAC: Authorizing `identity:get_project(project_id=1455a445f8e840e5ad8dcb7435d5871a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.510367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc17ac5-ae7b-4560-8242-468932627dc1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.511992 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 411/830] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/projects/1455a445f8e840e5ad8dcb7435d5871a => generated 311 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:29.518206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532adce6-3000-4ffc-8b2d-fce8e85e0994 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.619437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532adce6-3000-4ffc-8b2d-fce8e85e0994 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.655866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc806d-5984-45a4-aa8a-f59fb4d4974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.656528 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc806d-5984-45a4-aa8a-f59fb4d4974f None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.656836 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc806d-5984-45a4-aa8a-f59fb4d4974f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.657111 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc806d-5984-45a4-aa8a-f59fb4d4974f None admin] PATH_INFO: `/v3/projects/1455a445f8e840e5ad8dcb7435d5871a/users/0160f96dd8de4e5eb683bc4888c55e6b/roles/81d287a8b606458d87c9607e8b48ac58` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.659555 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc806d-5984-45a4-aa8a-f59fb4d4974f None admin] RBAC: Authorizing `identity:create_grant(project_id=1455a445f8e840e5ad8dcb7435d5871a, user_id=0160f96dd8de4e5eb683bc4888c55e6b, role_id=81d287a8b606458d87c9607e8b48ac58)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.661030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc806d-5984-45a4-aa8a-f59fb4d4974f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.674607 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 420/831] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:29 2022] PUT /identity/v3/projects/1455a445f8e840e5ad8dcb7435d5871a/users/0160f96dd8de4e5eb683bc4888c55e6b/roles/81d287a8b606458d87c9607e8b48ac58 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:31.762147 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067b0ef-5212-4f96-a95a-b78366feebaf None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.762658 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067b0ef-5212-4f96-a95a-b78366feebaf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.762763 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067b0ef-5212-4f96-a95a-b78366feebaf None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.764015 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 412/832] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:31.769917 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74392d9-69cd-4dad-9c23-e960c6cbcb98 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.770338 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74392d9-69cd-4dad-9c23-e960c6cbcb98 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.770468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74392d9-69cd-4dad-9c23-e960c6cbcb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.788683 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a74392d9-69cd-4dad-9c23-e960c6cbcb98 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:31.789869 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a74392d9-69cd-4dad-9c23-e960c6cbcb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:31.812524 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:31.855446 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 421/833] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:15:32.119837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067b0ef-5212-4f96-a95a-b78366feebaf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.218325 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067b0ef-5212-4f96-a95a-b78366feebaf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.251472 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec9546-6719-414a-ad17-32ec6c5eebb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.252088 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec9546-6719-414a-ad17-32ec6c5eebb0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.252369 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec9546-6719-414a-ad17-32ec6c5eebb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.252673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec9546-6719-414a-ad17-32ec6c5eebb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.262553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec9546-6719-414a-ad17-32ec6c5eebb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-70ec9546-6719-414a-ad17-32ec6c5eebb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:32.267835 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:32.268955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:32.269957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 413/834] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:32.274869 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74392d9-69cd-4dad-9c23-e960c6cbcb98 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.372938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74392d9-69cd-4dad-9c23-e960c6cbcb98 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.519883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd497fd-7014-4fb5-a2f3-6dd38d340e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.520489 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd497fd-7014-4fb5-a2f3-6dd38d340e0d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.520799 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd497fd-7014-4fb5-a2f3-6dd38d340e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.521073 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd497fd-7014-4fb5-a2f3-6dd38d340e0d None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.521728 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd497fd-7014-4fb5-a2f3-6dd38d340e0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:32.522029 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd497fd-7014-4fb5-a2f3-6dd38d340e0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.523380 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd497fd-7014-4fb5-a2f3-6dd38d340e0d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.531245 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 422/835] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:32.538448 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec9546-6719-414a-ad17-32ec6c5eebb0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.562179 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.633609 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec9546-6719-414a-ad17-32ec6c5eebb0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.666221 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee571e-363b-4821-a427-16b7ac588f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.666839 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee571e-363b-4821-a427-16b7ac588f8e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.667112 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee571e-363b-4821-a427-16b7ac588f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.667378 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee571e-363b-4821-a427-16b7ac588f8e None admin] PATH_INFO: `/v3/users/0160f96dd8de4e5eb683bc4888c55e6b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.668467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee571e-363b-4821-a427-16b7ac588f8e None admin] RBAC: Authorizing `identity:get_user(user_id=0160f96dd8de4e5eb683bc4888c55e6b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.669933 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee571e-363b-4821-a427-16b7ac588f8e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.671463 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 414/836] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/users/0160f96dd8de4e5eb683bc4888c55e6b => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:32.677376 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd497fd-7014-4fb5-a2f3-6dd38d340e0d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.701475 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.773791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd497fd-7014-4fb5-a2f3-6dd38d340e0d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.805226 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbfe9b-1ca1-42be-b5c6-ba6c9c645321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.805807 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbfe9b-1ca1-42be-b5c6-ba6c9c645321 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.806080 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbfe9b-1ca1-42be-b5c6-ba6c9c645321 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.806347 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbfe9b-1ca1-42be-b5c6-ba6c9c645321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.870376 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bbfe9b-1ca1-42be-b5c6-ba6c9c645321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.871780 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bbfe9b-1ca1-42be-b5c6-ba6c9c645321 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.893263 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.965019 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 423/837] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:15:32.971390 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee571e-363b-4821-a427-16b7ac588f8e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.059547 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:33.068382 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee571e-363b-4821-a427-16b7ac588f8e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.101638 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9ae6c-f846-4c3b-a1ad-30c440b173b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.102275 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9ae6c-f846-4c3b-a1ad-30c440b173b4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.102550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9ae6c-f846-4c3b-a1ad-30c440b173b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.102817 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9ae6c-f846-4c3b-a1ad-30c440b173b4 None admin] PATH_INFO: `/v3/projects/1455a445f8e840e5ad8dcb7435d5871a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.103897 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9ae6c-f846-4c3b-a1ad-30c440b173b4 None admin] RBAC: Authorizing `identity:get_project(project_id=1455a445f8e840e5ad8dcb7435d5871a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.105333 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9ae6c-f846-4c3b-a1ad-30c440b173b4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.106895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 415/838] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/projects/1455a445f8e840e5ad8dcb7435d5871a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:33.112874 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbfe9b-1ca1-42be-b5c6-ba6c9c645321 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:33.207085 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbfe9b-1ca1-42be-b5c6-ba6c9c645321 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:33.228888 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:33.239840 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63144c7-14a9-4735-b6d8-c2af69af5e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:33.240429 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63144c7-14a9-4735-b6d8-c2af69af5e1a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:33.240730 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63144c7-14a9-4735-b6d8-c2af69af5e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:33.240992 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63144c7-14a9-4735-b6d8-c2af69af5e1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:33.241697 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63144c7-14a9-4735-b6d8-c2af69af5e1a None admin] RBAC: Adding query filter params (role.id=81d287a8b606458d87c9607e8b48ac58, scope.project.id=1455a445f8e840e5ad8dcb7435d5871a, user.id=0160f96dd8de4e5eb683bc4888c55e6b) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:33.241998 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63144c7-14a9-4735-b6d8-c2af69af5e1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:33.243387 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63144c7-14a9-4735-b6d8-c2af69af5e1a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:33.248863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 424/839] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:33 2022] GET /identity/v3/role_assignments?user.id=0160f96dd8de4e5eb683bc4888c55e6b&scope.project.id=1455a445f8e840e5ad8dcb7435d5871a&role.id=81d287a8b606458d87c9607e8b48ac58 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:35.390066 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5512f6-5058-4003-81db-10c06b3dec82 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.390593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5512f6-5058-4003-81db-10c06b3dec82 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.390593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5512f6-5058-4003-81db-10c06b3dec82 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.391919 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 416/840] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:35.397178 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8af1c-f65a-41c8-b338-14f93f25e92c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.397609 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8af1c-f65a-41c8-b338-14f93f25e92c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.397701 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8af1c-f65a-41c8-b338-14f93f25e92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.415280 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-55a8af1c-f65a-41c8-b338-14f93f25e92c None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:35.416455 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-55a8af1c-f65a-41c8-b338-14f93f25e92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:35.480202 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 425/841] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:35.750457 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5512f6-5058-4003-81db-10c06b3dec82 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:35.849187 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5512f6-5058-4003-81db-10c06b3dec82 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:35.882415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62725d7e-948c-428f-9ac7-60a69afd5a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:35.883019 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725d7e-948c-428f-9ac7-60a69afd5a36 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.883296 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725d7e-948c-428f-9ac7-60a69afd5a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.883563 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725d7e-948c-428f-9ac7-60a69afd5a36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.884674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62725d7e-948c-428f-9ac7-60a69afd5a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:35.886072 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62725d7e-948c-428f-9ac7-60a69afd5a36 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:35.887576 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 417/842] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:35.893243 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8af1c-f65a-41c8-b338-14f93f25e92c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:35.987464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8af1c-f65a-41c8-b338-14f93f25e92c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:36.003493 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:36.021021 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7085a-ca07-4bfc-9e2f-27ba347a15bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:36.021609 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7085a-ca07-4bfc-9e2f-27ba347a15bf None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.021881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7085a-ca07-4bfc-9e2f-27ba347a15bf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.022145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7085a-ca07-4bfc-9e2f-27ba347a15bf None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.022785 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7085a-ca07-4bfc-9e2f-27ba347a15bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:36.024170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7085a-ca07-4bfc-9e2f-27ba347a15bf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:36.045135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 426/843] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:35 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:38.179858 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd243a-efaf-493f-8ec5-58452f88c0a6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.180401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd243a-efaf-493f-8ec5-58452f88c0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.180401 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd243a-efaf-493f-8ec5-58452f88c0a6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.181754 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 418/844] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:38.186911 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd48eee-b4b9-4ed0-a187-d5ca41c2c8bb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.187269 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd48eee-b4b9-4ed0-a187-d5ca41c2c8bb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.187448 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd48eee-b4b9-4ed0-a187-d5ca41c2c8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.206087 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2fd48eee-b4b9-4ed0-a187-d5ca41c2c8bb None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:38.207257 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2fd48eee-b4b9-4ed0-a187-d5ca41c2c8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:38.271371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 427/845] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:38.541391 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd243a-efaf-493f-8ec5-58452f88c0a6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.569455 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:38.641031 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd243a-efaf-493f-8ec5-58452f88c0a6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.674639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac377-2b5d-41e7-8f78-f442fbdb1e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.675260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac377-2b5d-41e7-8f78-f442fbdb1e9c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.675539 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac377-2b5d-41e7-8f78-f442fbdb1e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.675810 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac377-2b5d-41e7-8f78-f442fbdb1e9c None admin] PATH_INFO: `/v3/roles/1255151437d94364acba841f4215321c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.677212 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ac377-2b5d-41e7-8f78-f442fbdb1e9c None admin] RBAC: Authorizing `identity:get_role(role_id=1255151437d94364acba841f4215321c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:38.678577 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ac377-2b5d-41e7-8f78-f442fbdb1e9c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:38.679875 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 419/846] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/roles/1255151437d94364acba841f4215321c => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:38.685728 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd48eee-b4b9-4ed0-a187-d5ca41c2c8bb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.779061 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd48eee-b4b9-4ed0-a187-d5ca41c2c8bb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.780520 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:38.814305 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d93542-1e24-46a9-9a57-f28ca1b51dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.814868 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d93542-1e24-46a9-9a57-f28ca1b51dae None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.815141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d93542-1e24-46a9-9a57-f28ca1b51dae None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.815410 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d93542-1e24-46a9-9a57-f28ca1b51dae None admin] PATH_INFO: `/v3/users/a54a39e7a1de4763a76448a80caa8907` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.827770 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d93542-1e24-46a9-9a57-f28ca1b51dae None admin] RBAC: Authorizing `identity:get_user(user_id=a54a39e7a1de4763a76448a80caa8907)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:38.829268 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d93542-1e24-46a9-9a57-f28ca1b51dae None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:38.830828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 428/847] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/users/a54a39e7a1de4763a76448a80caa8907 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:38.836601 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac377-2b5d-41e7-8f78-f442fbdb1e9c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.930681 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:38.936884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac377-2b5d-41e7-8f78-f442fbdb1e9c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.973807 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc8a7b-8b5a-419a-b23e-e00af856134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.974426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc8a7b-8b5a-419a-b23e-e00af856134a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.974707 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc8a7b-8b5a-419a-b23e-e00af856134a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.974982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc8a7b-8b5a-419a-b23e-e00af856134a None admin] PATH_INFO: `/v3/projects/1455a445f8e840e5ad8dcb7435d5871a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.976079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc8a7b-8b5a-419a-b23e-e00af856134a None admin] RBAC: Authorizing `identity:get_project(project_id=1455a445f8e840e5ad8dcb7435d5871a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:38.977522 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc8a7b-8b5a-419a-b23e-e00af856134a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:38.979124 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 420/848] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/projects/1455a445f8e840e5ad8dcb7435d5871a => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:38.985654 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d93542-1e24-46a9-9a57-f28ca1b51dae None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:39.080717 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d93542-1e24-46a9-9a57-f28ca1b51dae None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:39.113632 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12a3a6-4188-4d8e-a89b-ca10ac9260a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:39.114270 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12a3a6-4188-4d8e-a89b-ca10ac9260a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.114547 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12a3a6-4188-4d8e-a89b-ca10ac9260a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.114818 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12a3a6-4188-4d8e-a89b-ca10ac9260a6 None admin] PATH_INFO: `/v3/projects/1455a445f8e840e5ad8dcb7435d5871a/users/a54a39e7a1de4763a76448a80caa8907/roles/1255151437d94364acba841f4215321c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.116947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12a3a6-4188-4d8e-a89b-ca10ac9260a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1455a445f8e840e5ad8dcb7435d5871a, user_id=a54a39e7a1de4763a76448a80caa8907, role_id=1255151437d94364acba841f4215321c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:39.118359 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12a3a6-4188-4d8e-a89b-ca10ac9260a6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:39.130647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 429/849] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:38 2022] PUT /identity/v3/projects/1455a445f8e840e5ad8dcb7435d5871a/users/a54a39e7a1de4763a76448a80caa8907/roles/1255151437d94364acba841f4215321c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:41.221024 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26feea-e62d-4c98-b987-f590061e6733 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.221547 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26feea-e62d-4c98-b987-f590061e6733 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.221547 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26feea-e62d-4c98-b987-f590061e6733 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.222888 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 421/850] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:41.228048 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf34f7-a05b-4236-b57c-aa685be53d44 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.228416 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf34f7-a05b-4236-b57c-aa685be53d44 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.228639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf34f7-a05b-4236-b57c-aa685be53d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.246888 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5dbf34f7-a05b-4236-b57c-aa685be53d44 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:41.248073 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5dbf34f7-a05b-4236-b57c-aa685be53d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:41.261715 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.313892 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 430/851] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:41.580162 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26feea-e62d-4c98-b987-f590061e6733 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.678392 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26feea-e62d-4c98-b987-f590061e6733 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.711296 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5fa5-9c3e-4b90-a212-93c773360f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.712168 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5fa5-9c3e-4b90-a212-93c773360f45 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.712447 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5fa5-9c3e-4b90-a212-93c773360f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.712749 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5fa5-9c3e-4b90-a212-93c773360f45 None admin] PATH_INFO: `/v3/roles/1255151437d94364acba841f4215321c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.714129 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc5fa5-9c3e-4b90-a212-93c773360f45 None admin] RBAC: Authorizing `identity:get_role(role_id=1255151437d94364acba841f4215321c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.715490 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc5fa5-9c3e-4b90-a212-93c773360f45 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.716821 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 422/852] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/roles/1255151437d94364acba841f4215321c => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.722620 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf34f7-a05b-4236-b57c-aa685be53d44 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.816772 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf34f7-a05b-4236-b57c-aa685be53d44 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.849450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccb1c1-7f9d-4704-839f-bafb469c7146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.850009 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccb1c1-7f9d-4704-839f-bafb469c7146 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.850290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccb1c1-7f9d-4704-839f-bafb469c7146 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.850562 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccb1c1-7f9d-4704-839f-bafb469c7146 None admin] PATH_INFO: `/v3/users/a54a39e7a1de4763a76448a80caa8907` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.851677 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ccb1c1-7f9d-4704-839f-bafb469c7146 None admin] RBAC: Authorizing `identity:get_user(user_id=a54a39e7a1de4763a76448a80caa8907)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.853114 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ccb1c1-7f9d-4704-839f-bafb469c7146 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.854662 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 431/853] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/users/a54a39e7a1de4763a76448a80caa8907 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.860152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5fa5-9c3e-4b90-a212-93c773360f45 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.890504 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.956988 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5fa5-9c3e-4b90-a212-93c773360f45 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.990420 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4df39-0ac2-4b2e-be4b-f4d32b6bb6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.991013 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4df39-0ac2-4b2e-be4b-f4d32b6bb6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.991293 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4df39-0ac2-4b2e-be4b-f4d32b6bb6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.991565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4df39-0ac2-4b2e-be4b-f4d32b6bb6a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.055666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4df39-0ac2-4b2e-be4b-f4d32b6bb6a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.057129 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4df39-0ac2-4b2e-be4b-f4d32b6bb6a3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.085380 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.151570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 423/854] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:15:42.157401 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccb1c1-7f9d-4704-839f-bafb469c7146 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.175880 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.253465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccb1c1-7f9d-4704-839f-bafb469c7146 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.285253 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57114fdc-166d-4212-b969-0669c655e470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.285866 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57114fdc-166d-4212-b969-0669c655e470 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.286140 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57114fdc-166d-4212-b969-0669c655e470 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.286406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57114fdc-166d-4212-b969-0669c655e470 None admin] PATH_INFO: `/v3/projects/1455a445f8e840e5ad8dcb7435d5871a` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.287491 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57114fdc-166d-4212-b969-0669c655e470 None admin] RBAC: Authorizing `identity:get_project(project_id=1455a445f8e840e5ad8dcb7435d5871a)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.288922 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57114fdc-166d-4212-b969-0669c655e470 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.290509 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 432/855] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/projects/1455a445f8e840e5ad8dcb7435d5871a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:42.296923 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4df39-0ac2-4b2e-be4b-f4d32b6bb6a3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.393586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4df39-0ac2-4b2e-be4b-f4d32b6bb6a3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.421996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.427745 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bb851-3319-4e12-b64a-6c41fe4c2986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.428340 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bb851-3319-4e12-b64a-6c41fe4c2986 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.428648 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bb851-3319-4e12-b64a-6c41fe4c2986 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.428924 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bb851-3319-4e12-b64a-6c41fe4c2986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.429640 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bb851-3319-4e12-b64a-6c41fe4c2986 None admin] RBAC: Adding query filter params (role.id=1255151437d94364acba841f4215321c, scope.project.id=1455a445f8e840e5ad8dcb7435d5871a, user.id=a54a39e7a1de4763a76448a80caa8907) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.429947 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bb851-3319-4e12-b64a-6c41fe4c2986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.431354 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bb851-3319-4e12-b64a-6c41fe4c2986 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.436991 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 424/856] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/role_assignments?user.id=a54a39e7a1de4763a76448a80caa8907&scope.project.id=1455a445f8e840e5ad8dcb7435d5871a&role.id=1255151437d94364acba841f4215321c => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:44.555977 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89f801-eab6-4d93-aa72-2b53518bd126 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.556502 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89f801-eab6-4d93-aa72-2b53518bd126 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.556502 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89f801-eab6-4d93-aa72-2b53518bd126 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.557887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 433/857] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:44.562979 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37a09b-0c70-4c01-aa19-47f20bc37f5d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.563896 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37a09b-0c70-4c01-aa19-47f20bc37f5d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.564098 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37a09b-0c70-4c01-aa19-47f20bc37f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.581740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1c37a09b-0c70-4c01-aa19-47f20bc37f5d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:44.582908 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1c37a09b-0c70-4c01-aa19-47f20bc37f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:44.647751 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 425/858] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:44.912213 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89f801-eab6-4d93-aa72-2b53518bd126 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:44.994141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:45.009962 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89f801-eab6-4d93-aa72-2b53518bd126 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:45.041688 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d930ac-46a4-435f-becc-b888e9abc33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:45.042304 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d930ac-46a4-435f-becc-b888e9abc33c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:45.042577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d930ac-46a4-435f-becc-b888e9abc33c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:45.042845 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d930ac-46a4-435f-becc-b888e9abc33c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:45.043930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d930ac-46a4-435f-becc-b888e9abc33c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:45.045364 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d930ac-46a4-435f-becc-b888e9abc33c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:45.046903 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 434/859] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:45.053257 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37a09b-0c70-4c01-aa19-47f20bc37f5d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:45.147290 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37a09b-0c70-4c01-aa19-47f20bc37f5d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:45.171359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:45.180390 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed950d4-261c-46ce-a8e8-a610c686d561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:45.181013 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed950d4-261c-46ce-a8e8-a610c686d561 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:45.181292 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed950d4-261c-46ce-a8e8-a610c686d561 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:45.181561 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed950d4-261c-46ce-a8e8-a610c686d561 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:45.182193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed950d4-261c-46ce-a8e8-a610c686d561 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:45.183601 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed950d4-261c-46ce-a8e8-a610c686d561 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:45.208744 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 426/860] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:15:45 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:47.366171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec51de-bc69-4a29-8d52-2ba26aa388e4 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.366719 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec51de-bc69-4a29-8d52-2ba26aa388e4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.366719 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec51de-bc69-4a29-8d52-2ba26aa388e4 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.368053 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 435/861] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:47.373440 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85872f6c-a7d8-4b59-bff9-7707d5e6e299 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.373844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85872f6c-a7d8-4b59-bff9-7707d5e6e299 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.373995 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85872f6c-a7d8-4b59-bff9-7707d5e6e299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.392171 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-85872f6c-a7d8-4b59-bff9-7707d5e6e299 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:47.393377 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-85872f6c-a7d8-4b59-bff9-7707d5e6e299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:47.457747 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 427/862] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:47.722274 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec51de-bc69-4a29-8d52-2ba26aa388e4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.818257 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec51de-bc69-4a29-8d52-2ba26aa388e4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.850321 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dd671-019d-40cf-a91d-2df165452a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.850931 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dd671-019d-40cf-a91d-2df165452a3b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.851200 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dd671-019d-40cf-a91d-2df165452a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.851464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dd671-019d-40cf-a91d-2df165452a3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.852561 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416dd671-019d-40cf-a91d-2df165452a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.853962 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416dd671-019d-40cf-a91d-2df165452a3b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.855477 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 436/863] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.861552 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85872f6c-a7d8-4b59-bff9-7707d5e6e299 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.955638 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85872f6c-a7d8-4b59-bff9-7707d5e6e299 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.964666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.990661 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340ce06-0d7f-478d-bc34-ea45e4b30ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.991260 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340ce06-0d7f-478d-bc34-ea45e4b30ff9 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.991540 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340ce06-0d7f-478d-bc34-ea45e4b30ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.991811 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340ce06-0d7f-478d-bc34-ea45e4b30ff9 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.992460 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340ce06-0d7f-478d-bc34-ea45e4b30ff9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.993907 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340ce06-0d7f-478d-bc34-ea45e4b30ff9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:48.014089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 428/864] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:47 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:50.173052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7685c2-19d0-4f5a-b57c-ca08c51973f9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.173555 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7685c2-19d0-4f5a-b57c-ca08c51973f9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.173670 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7685c2-19d0-4f5a-b57c-ca08c51973f9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.174924 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 437/865] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:50.179955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c224266-d398-465f-8983-cc7cd4c963ec None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.180356 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c224266-d398-465f-8983-cc7cd4c963ec None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.180491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c224266-d398-465f-8983-cc7cd4c963ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.198614 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0c224266-d398-465f-8983-cc7cd4c963ec None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:50.199779 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0c224266-d398-465f-8983-cc7cd4c963ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:50.263671 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 429/866] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:50.529116 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7685c2-19d0-4f5a-b57c-ca08c51973f9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.553375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.628697 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7685c2-19d0-4f5a-b57c-ca08c51973f9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.661378 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f788d99-9a41-499c-8d2b-d6dd429d9e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.661999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f788d99-9a41-499c-8d2b-d6dd429d9e8b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.662274 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f788d99-9a41-499c-8d2b-d6dd429d9e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.662537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f788d99-9a41-499c-8d2b-d6dd429d9e8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.672157 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f788d99-9a41-499c-8d2b-d6dd429d9e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2f788d99-9a41-499c-8d2b-d6dd429d9e8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:50.676940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:50.678018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:15:50.679057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 438/867] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:50.683942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c224266-d398-465f-8983-cc7cd4c963ec None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.773227 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.782035 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c224266-d398-465f-8983-cc7cd4c963ec None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.815036 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be9aa5-98f9-4a93-a772-5b6209a7ab09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.815631 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be9aa5-98f9-4a93-a772-5b6209a7ab09 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.815912 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be9aa5-98f9-4a93-a772-5b6209a7ab09 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.816187 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be9aa5-98f9-4a93-a772-5b6209a7ab09 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.816872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be9aa5-98f9-4a93-a772-5b6209a7ab09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:50.817178 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be9aa5-98f9-4a93-a772-5b6209a7ab09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.818540 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be9aa5-98f9-4a93-a772-5b6209a7ab09 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.826209 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 430/868] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:50.832363 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f788d99-9a41-499c-8d2b-d6dd429d9e8b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.911347 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.929388 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f788d99-9a41-499c-8d2b-d6dd429d9e8b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.963000 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70189ac-acb9-4857-8e85-256857ff8912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.963613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70189ac-acb9-4857-8e85-256857ff8912 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.963895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70189ac-acb9-4857-8e85-256857ff8912 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.964170 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70189ac-acb9-4857-8e85-256857ff8912 None admin] PATH_INFO: `/v3/users/f3af0da4dc1746faa55c55ed0198cca4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.976781 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70189ac-acb9-4857-8e85-256857ff8912 None admin] RBAC: Authorizing `identity:get_user(user_id=f3af0da4dc1746faa55c55ed0198cca4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.978203 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70189ac-acb9-4857-8e85-256857ff8912 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.979769 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 439/869] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/users/f3af0da4dc1746faa55c55ed0198cca4 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:50.985931 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be9aa5-98f9-4a93-a772-5b6209a7ab09 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.081389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be9aa5-98f9-4a93-a772-5b6209a7ab09 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.113799 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13ee43-621a-4931-9dfe-37074a2ce0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.114420 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13ee43-621a-4931-9dfe-37074a2ce0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.114697 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13ee43-621a-4931-9dfe-37074a2ce0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.114969 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13ee43-621a-4931-9dfe-37074a2ce0b5 None admin] PATH_INFO: `/v3/projects/3eb296f4ed5c458e856ebd0ccbf77a74` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.116063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13ee43-621a-4931-9dfe-37074a2ce0b5 None admin] RBAC: Authorizing `identity:get_project(project_id=3eb296f4ed5c458e856ebd0ccbf77a74)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.117499 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13ee43-621a-4931-9dfe-37074a2ce0b5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.119093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 431/870] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/projects/3eb296f4ed5c458e856ebd0ccbf77a74 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:51.125039 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70189ac-acb9-4857-8e85-256857ff8912 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.221342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70189ac-acb9-4857-8e85-256857ff8912 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.253969 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c728cfd-ea30-41dd-92df-392ceab4229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.254608 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c728cfd-ea30-41dd-92df-392ceab4229a None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.254885 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c728cfd-ea30-41dd-92df-392ceab4229a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.255156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c728cfd-ea30-41dd-92df-392ceab4229a None admin] PATH_INFO: `/v3/projects/3eb296f4ed5c458e856ebd0ccbf77a74/users/f3af0da4dc1746faa55c55ed0198cca4/roles/81d287a8b606458d87c9607e8b48ac58` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.257037 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c728cfd-ea30-41dd-92df-392ceab4229a None admin] RBAC: Authorizing `identity:create_grant(project_id=3eb296f4ed5c458e856ebd0ccbf77a74, user_id=f3af0da4dc1746faa55c55ed0198cca4, role_id=81d287a8b606458d87c9607e8b48ac58)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.258470 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c728cfd-ea30-41dd-92df-392ceab4229a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.267451 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.272863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 440/871] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:51 2022] PUT /identity/v3/projects/3eb296f4ed5c458e856ebd0ccbf77a74/users/f3af0da4dc1746faa55c55ed0198cca4/roles/81d287a8b606458d87c9607e8b48ac58 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:53.431018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51ac1-908c-4a04-833b-26cc8cf3b9e3 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.431549 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51ac1-908c-4a04-833b-26cc8cf3b9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.431549 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51ac1-908c-4a04-833b-26cc8cf3b9e3 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.432927 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 432/872] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:53.438152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d704e-5c66-42b7-8ea0-c6a25a215efd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.438577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d704e-5c66-42b7-8ea0-c6a25a215efd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.438680 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d704e-5c66-42b7-8ea0-c6a25a215efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.457269 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7b7d704e-5c66-42b7-8ea0-c6a25a215efd None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:53.458472 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7b7d704e-5c66-42b7-8ea0-c6a25a215efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:53.523956 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 441/873] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:53.791576 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa51ac1-908c-4a04-833b-26cc8cf3b9e3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.817144 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.891955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa51ac1-908c-4a04-833b-26cc8cf3b9e3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.923673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb766c7-9a59-4e78-98e2-41298427b831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.924293 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb766c7-9a59-4e78-98e2-41298427b831 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.924594 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb766c7-9a59-4e78-98e2-41298427b831 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.924865 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb766c7-9a59-4e78-98e2-41298427b831 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.935193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb766c7-9a59-4e78-98e2-41298427b831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4cb766c7-9a59-4e78-98e2-41298427b831 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:53.940467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:53.941545 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:15:53.942389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 433/874] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:53.947630 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d704e-5c66-42b7-8ea0-c6a25a215efd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.041806 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d704e-5c66-42b7-8ea0-c6a25a215efd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.069613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.075756 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d72dd-1a10-4b10-b673-6a3712511c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.076349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d72dd-1a10-4b10-b673-6a3712511c4e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.076650 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d72dd-1a10-4b10-b673-6a3712511c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.076920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d72dd-1a10-4b10-b673-6a3712511c4e None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.077575 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d72dd-1a10-4b10-b673-6a3712511c4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:54.077872 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d72dd-1a10-4b10-b673-6a3712511c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.079320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d72dd-1a10-4b10-b673-6a3712511c4e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.087304 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 442/875] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:54.093142 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb766c7-9a59-4e78-98e2-41298427b831 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.174483 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.190415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb766c7-9a59-4e78-98e2-41298427b831 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.221953 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be48cd6-0350-4759-a235-9f6b326cea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.222563 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be48cd6-0350-4759-a235-9f6b326cea5a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.222839 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be48cd6-0350-4759-a235-9f6b326cea5a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.223109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be48cd6-0350-4759-a235-9f6b326cea5a None admin] PATH_INFO: `/v3/users/f3af0da4dc1746faa55c55ed0198cca4` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.224191 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be48cd6-0350-4759-a235-9f6b326cea5a None admin] RBAC: Authorizing `identity:get_user(user_id=f3af0da4dc1746faa55c55ed0198cca4)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.225633 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be48cd6-0350-4759-a235-9f6b326cea5a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.227166 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 434/876] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/users/f3af0da4dc1746faa55c55ed0198cca4 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:54.232918 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d72dd-1a10-4b10-b673-6a3712511c4e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.327859 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d72dd-1a10-4b10-b673-6a3712511c4e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.359860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8127f-25b7-41e3-b8ae-83d9c99b32bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.360449 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8127f-25b7-41e3-b8ae-83d9c99b32bc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.360752 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8127f-25b7-41e3-b8ae-83d9c99b32bc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.361024 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8127f-25b7-41e3-b8ae-83d9c99b32bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.415871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.425392 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8127f-25b7-41e3-b8ae-83d9c99b32bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.426805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8127f-25b7-41e3-b8ae-83d9c99b32bc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.521867 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 443/877] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:15:54.527681 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be48cd6-0350-4759-a235-9f6b326cea5a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.624760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be48cd6-0350-4759-a235-9f6b326cea5a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.656258 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2afb2c-8ec5-46bb-a43c-f1f16f50e3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.657446 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2afb2c-8ec5-46bb-a43c-f1f16f50e3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.657723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2afb2c-8ec5-46bb-a43c-f1f16f50e3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.657992 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2afb2c-8ec5-46bb-a43c-f1f16f50e3e5 None admin] PATH_INFO: `/v3/projects/3eb296f4ed5c458e856ebd0ccbf77a74` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.659079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2afb2c-8ec5-46bb-a43c-f1f16f50e3e5 None admin] RBAC: Authorizing `identity:get_project(project_id=3eb296f4ed5c458e856ebd0ccbf77a74)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.660495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2afb2c-8ec5-46bb-a43c-f1f16f50e3e5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.662122 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 435/878] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/projects/3eb296f4ed5c458e856ebd0ccbf77a74 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:54.668137 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8127f-25b7-41e3-b8ae-83d9c99b32bc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.754404 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.763077 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8127f-25b7-41e3-b8ae-83d9c99b32bc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.795822 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bacef1-0789-41e6-8e28-36a0aa3acb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.796403 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bacef1-0789-41e6-8e28-36a0aa3acb3e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.796706 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bacef1-0789-41e6-8e28-36a0aa3acb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.796973 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bacef1-0789-41e6-8e28-36a0aa3acb3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.798194 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bacef1-0789-41e6-8e28-36a0aa3acb3e None admin] RBAC: Adding query filter params (role.id=81d287a8b606458d87c9607e8b48ac58, scope.project.id=3eb296f4ed5c458e856ebd0ccbf77a74, user.id=f3af0da4dc1746faa55c55ed0198cca4) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:54.798496 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bacef1-0789-41e6-8e28-36a0aa3acb3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.799896 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bacef1-0789-41e6-8e28-36a0aa3acb3e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.805411 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 444/879] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/role_assignments?user.id=f3af0da4dc1746faa55c55ed0198cca4&scope.project.id=3eb296f4ed5c458e856ebd0ccbf77a74&role.id=81d287a8b606458d87c9607e8b48ac58 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:56.923621 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1905-9594-46ee-9b0a-37ba2bc63c01 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.924165 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1905-9594-46ee-9b0a-37ba2bc63c01 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.924165 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1905-9594-46ee-9b0a-37ba2bc63c01 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.925427 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 436/880] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:56.930650 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10170fae-a1ca-4f58-bd8a-97481d59cae4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.931073 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10170fae-a1ca-4f58-bd8a-97481d59cae4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.931171 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10170fae-a1ca-4f58-bd8a-97481d59cae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.948855 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-10170fae-a1ca-4f58-bd8a-97481d59cae4 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:56.950027 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-10170fae-a1ca-4f58-bd8a-97481d59cae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:57.013653 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 445/881] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:57.280384 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f1905-9594-46ee-9b0a-37ba2bc63c01 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:57.302428 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:57.379903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f1905-9594-46ee-9b0a-37ba2bc63c01 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:57.412160 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898779f-7b03-4650-a510-17f15093f853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:57.412780 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898779f-7b03-4650-a510-17f15093f853 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.413059 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898779f-7b03-4650-a510-17f15093f853 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.413325 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898779f-7b03-4650-a510-17f15093f853 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.413972 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898779f-7b03-4650-a510-17f15093f853 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:57.415315 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898779f-7b03-4650-a510-17f15093f853 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:57.431507 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 437/882] 10.222.0.15 () {60 vars in 1138 bytes} [Wed Nov 2 07:15:57 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:59.584953 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11569-3b9c-400d-a071-c1d5807d7ef5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.585519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11569-3b9c-400d-a071-c1d5807d7ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.585519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11569-3b9c-400d-a071-c1d5807d7ef5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.587093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 446/883] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:59.592255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38787f-2234-4854-adcd-a86ad42129fb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.592699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38787f-2234-4854-adcd-a86ad42129fb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.592808 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38787f-2234-4854-adcd-a86ad42129fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.611102 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0a38787f-2234-4854-adcd-a86ad42129fb None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:59.612278 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0a38787f-2234-4854-adcd-a86ad42129fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:59.676886 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 438/884] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:15:59.944566 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a11569-3b9c-400d-a071-c1d5807d7ef5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.026354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:00.042538 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a11569-3b9c-400d-a071-c1d5807d7ef5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.075458 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d01e0b-86ba-41d5-afc7-b2a72374e8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.076068 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d01e0b-86ba-41d5-afc7-b2a72374e8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.076346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d01e0b-86ba-41d5-afc7-b2a72374e8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.076642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d01e0b-86ba-41d5-afc7-b2a72374e8b6 None admin] PATH_INFO: `/v3/domains/52aff1f8cf7c4404a923ad9d4aee838b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.086969 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d01e0b-86ba-41d5-afc7-b2a72374e8b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=52aff1f8cf7c4404a923ad9d4aee838b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.088380 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d01e0b-86ba-41d5-afc7-b2a72374e8b6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.089969 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 447/885] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/domains/52aff1f8cf7c4404a923ad9d4aee838b => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:00.095649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38787f-2234-4854-adcd-a86ad42129fb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.106900 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:00.191720 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38787f-2234-4854-adcd-a86ad42129fb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.223925 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb7f42-b783-443d-bddf-4f2ea9213bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.224473 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb7f42-b783-443d-bddf-4f2ea9213bfc None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.224780 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb7f42-b783-443d-bddf-4f2ea9213bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.225054 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb7f42-b783-443d-bddf-4f2ea9213bfc None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.225691 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb7f42-b783-443d-bddf-4f2ea9213bfc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.227098 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb7f42-b783-443d-bddf-4f2ea9213bfc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.253127 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 439/886] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:00 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:02.427918 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecae5f-f83a-498d-b1e1-f4ae50c63192 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.428482 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecae5f-f83a-498d-b1e1-f4ae50c63192 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.428482 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecae5f-f83a-498d-b1e1-f4ae50c63192 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.429824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 448/887] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:02.435239 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a00ec1-b071-47fd-aac7-6bf3c48da247 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.435693 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a00ec1-b071-47fd-aac7-6bf3c48da247 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.435785 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a00ec1-b071-47fd-aac7-6bf3c48da247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.457515 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-93a00ec1-b071-47fd-aac7-6bf3c48da247 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:02.458717 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-93a00ec1-b071-47fd-aac7-6bf3c48da247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:02.524505 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:02.536243 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 440/888] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:02.803408 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecae5f-f83a-498d-b1e1-f4ae50c63192 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:02.914457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecae5f-f83a-498d-b1e1-f4ae50c63192 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:02.946179 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ce12e-683b-4f30-a4ed-80686c1ab395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:02.946793 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ce12e-683b-4f30-a4ed-80686c1ab395 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.947069 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ce12e-683b-4f30-a4ed-80686c1ab395 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.947339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ce12e-683b-4f30-a4ed-80686c1ab395 None admin] PATH_INFO: `/v3/domains/52aff1f8cf7c4404a923ad9d4aee838b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.948424 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ce12e-683b-4f30-a4ed-80686c1ab395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=52aff1f8cf7c4404a923ad9d4aee838b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:02.949860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ce12e-683b-4f30-a4ed-80686c1ab395 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:02.951406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 449/889] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Nov 2 07:16:02 2022] GET /identity/v3/domains/52aff1f8cf7c4404a923ad9d4aee838b => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:02.957816 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a00ec1-b071-47fd-aac7-6bf3c48da247 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.053168 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a00ec1-b071-47fd-aac7-6bf3c48da247 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.093706 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068883ac-b62b-4c20-be0e-765486a76bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.094781 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068883ac-b62b-4c20-be0e-765486a76bc4 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.095056 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068883ac-b62b-4c20-be0e-765486a76bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.095323 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068883ac-b62b-4c20-be0e-765486a76bc4 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.095980 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068883ac-b62b-4c20-be0e-765486a76bc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.097425 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068883ac-b62b-4c20-be0e-765486a76bc4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.118742 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 441/890] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:02 2022] POST /identity/v3/users => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:05.285818 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76a068-5fc1-4731-a16f-b82fa22f749f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.288359 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76a068-5fc1-4731-a16f-b82fa22f749f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.288359 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76a068-5fc1-4731-a16f-b82fa22f749f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.288359 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 450/891] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:05.293159 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1d0a6-f16b-43fe-85ea-f159fcf398a0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.293739 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1d0a6-f16b-43fe-85ea-f159fcf398a0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.293739 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1d0a6-f16b-43fe-85ea-f159fcf398a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.312352 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-42c1d0a6-f16b-43fe-85ea-f159fcf398a0 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:05.314639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-42c1d0a6-f16b-43fe-85ea-f159fcf398a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:05.356362 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:05.380553 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 442/892] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:05.646092 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76a068-5fc1-4731-a16f-b82fa22f749f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.662370 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:05.745175 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76a068-5fc1-4731-a16f-b82fa22f749f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:05.776556 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ff3ac-8711-4366-b887-bfd45038fdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:05.777177 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ff3ac-8711-4366-b887-bfd45038fdfb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.777457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ff3ac-8711-4366-b887-bfd45038fdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.777724 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ff3ac-8711-4366-b887-bfd45038fdfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.788308 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ff3ac-8711-4366-b887-bfd45038fdfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-840ff3ac-8711-4366-b887-bfd45038fdfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:05.793097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:05.794252 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:16:05.795382 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 451/893] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:05.800770 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1d0a6-f16b-43fe-85ea-f159fcf398a0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.895193 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1d0a6-f16b-43fe-85ea-f159fcf398a0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:05.928733 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9936b-389b-4f2d-8260-144b09dd09ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:05.929330 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9936b-389b-4f2d-8260-144b09dd09ab None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.929607 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9936b-389b-4f2d-8260-144b09dd09ab None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.929878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9936b-389b-4f2d-8260-144b09dd09ab None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.930530 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9936b-389b-4f2d-8260-144b09dd09ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:05.930828 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9936b-389b-4f2d-8260-144b09dd09ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:05.932181 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9936b-389b-4f2d-8260-144b09dd09ab None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:05.939869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 443/894] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:05.945624 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ff3ac-8711-4366-b887-bfd45038fdfb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.018054 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.042711 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ff3ac-8711-4366-b887-bfd45038fdfb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.074293 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d4554-a20a-4f49-8026-23d78f8e31ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.074907 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d4554-a20a-4f49-8026-23d78f8e31ad None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.075188 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d4554-a20a-4f49-8026-23d78f8e31ad None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.075464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d4554-a20a-4f49-8026-23d78f8e31ad None admin] PATH_INFO: `/v3/users/27de1ce6475948f0b2817b31b881ff9b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.089662 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d4554-a20a-4f49-8026-23d78f8e31ad None admin] RBAC: Authorizing `identity:get_user(user_id=27de1ce6475948f0b2817b31b881ff9b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.091083 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d4554-a20a-4f49-8026-23d78f8e31ad None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.092664 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 452/895] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/users/27de1ce6475948f0b2817b31b881ff9b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:06.098232 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9936b-389b-4f2d-8260-144b09dd09ab None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.140301 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.196707 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9936b-389b-4f2d-8260-144b09dd09ab None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.230148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f54c7-bee7-4636-b8c6-3933a2178cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.230774 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f54c7-bee7-4636-b8c6-3933a2178cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.231057 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f54c7-bee7-4636-b8c6-3933a2178cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.231332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f54c7-bee7-4636-b8c6-3933a2178cb7 None admin] PATH_INFO: `/v3/projects/fbc389afc41e48b7b4bfa4d91dde83c0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.232434 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f54c7-bee7-4636-b8c6-3933a2178cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc389afc41e48b7b4bfa4d91dde83c0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.233906 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f54c7-bee7-4636-b8c6-3933a2178cb7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.235557 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 444/896] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/projects/fbc389afc41e48b7b4bfa4d91dde83c0 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:06.241436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d4554-a20a-4f49-8026-23d78f8e31ad None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.337354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d4554-a20a-4f49-8026-23d78f8e31ad None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.366228 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.371300 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e1368-9677-4f0a-a60f-f5d506e61caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.371948 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e1368-9677-4f0a-a60f-f5d506e61caa None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.372229 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e1368-9677-4f0a-a60f-f5d506e61caa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.372504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e1368-9677-4f0a-a60f-f5d506e61caa None admin] PATH_INFO: `/v3/projects/fbc389afc41e48b7b4bfa4d91dde83c0/users/27de1ce6475948f0b2817b31b881ff9b/roles/81d287a8b606458d87c9607e8b48ac58` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.374418 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e1368-9677-4f0a-a60f-f5d506e61caa None admin] RBAC: Authorizing `identity:create_grant(project_id=fbc389afc41e48b7b4bfa4d91dde83c0, user_id=27de1ce6475948f0b2817b31b881ff9b, role_id=81d287a8b606458d87c9607e8b48ac58)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.376361 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e1368-9677-4f0a-a60f-f5d506e61caa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.388230 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 453/897] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:06 2022] PUT /identity/v3/projects/fbc389afc41e48b7b4bfa4d91dde83c0/users/27de1ce6475948f0b2817b31b881ff9b/roles/81d287a8b606458d87c9607e8b48ac58 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:08.480858 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a17a8-469c-4d3a-82aa-09de0132db62 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.481415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a17a8-469c-4d3a-82aa-09de0132db62 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.481415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a17a8-469c-4d3a-82aa-09de0132db62 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.482706 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 445/898] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:08.488449 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751df4b8-57d0-4264-9206-90ec1f1ecd87 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.488927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751df4b8-57d0-4264-9206-90ec1f1ecd87 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.489024 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751df4b8-57d0-4264-9206-90ec1f1ecd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.507315 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-751df4b8-57d0-4264-9206-90ec1f1ecd87 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:08.508504 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-751df4b8-57d0-4264-9206-90ec1f1ecd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:08.572471 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 454/899] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:08.837006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a17a8-469c-4d3a-82aa-09de0132db62 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.933784 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a17a8-469c-4d3a-82aa-09de0132db62 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.942660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.967097 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a6ca3-4f4d-4cac-b700-007bac6f344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.967720 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6ca3-4f4d-4cac-b700-007bac6f344a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.967997 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6ca3-4f4d-4cac-b700-007bac6f344a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.968266 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6ca3-4f4d-4cac-b700-007bac6f344a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.978450 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a6ca3-4f4d-4cac-b700-007bac6f344a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-572a6ca3-4f4d-4cac-b700-007bac6f344a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:08.983177 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:08.984756 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:16:08.985585 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 446/900] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:08.990824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751df4b8-57d0-4264-9206-90ec1f1ecd87 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.085939 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751df4b8-57d0-4264-9206-90ec1f1ecd87 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.107622 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.122228 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00397441-8db0-429b-9b9b-8f68bd345594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.122826 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00397441-8db0-429b-9b9b-8f68bd345594 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.123105 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00397441-8db0-429b-9b9b-8f68bd345594 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.123376 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00397441-8db0-429b-9b9b-8f68bd345594 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.124058 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00397441-8db0-429b-9b9b-8f68bd345594 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:09.124475 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00397441-8db0-429b-9b9b-8f68bd345594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.126855 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00397441-8db0-429b-9b9b-8f68bd345594 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.135996 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 455/901] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:09.141936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a6ca3-4f4d-4cac-b700-007bac6f344a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.238359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a6ca3-4f4d-4cac-b700-007bac6f344a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.270396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefe99b-17a6-43b1-a2fd-245198ba14c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.271013 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefe99b-17a6-43b1-a2fd-245198ba14c5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.271292 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefe99b-17a6-43b1-a2fd-245198ba14c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.271564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefe99b-17a6-43b1-a2fd-245198ba14c5 None admin] PATH_INFO: `/v3/users/27de1ce6475948f0b2817b31b881ff9b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.272685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefe99b-17a6-43b1-a2fd-245198ba14c5 None admin] RBAC: Authorizing `identity:get_user(user_id=27de1ce6475948f0b2817b31b881ff9b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.274094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefe99b-17a6-43b1-a2fd-245198ba14c5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.275618 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 447/902] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/users/27de1ce6475948f0b2817b31b881ff9b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:09.281325 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00397441-8db0-429b-9b9b-8f68bd345594 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.378733 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00397441-8db0-429b-9b9b-8f68bd345594 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.410668 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ab404-2558-43a3-a9af-91cebf29cd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.411282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ab404-2558-43a3-a9af-91cebf29cd06 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.411541 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ab404-2558-43a3-a9af-91cebf29cd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.411813 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ab404-2558-43a3-a9af-91cebf29cd06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.460498 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.476317 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ab404-2558-43a3-a9af-91cebf29cd06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.477772 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ab404-2558-43a3-a9af-91cebf29cd06 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.570727 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 456/903] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:16:09.576567 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefe99b-17a6-43b1-a2fd-245198ba14c5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.611717 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.674514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefe99b-17a6-43b1-a2fd-245198ba14c5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.706726 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a637339-2903-4973-8087-ad0a149ff5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.707351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a637339-2903-4973-8087-ad0a149ff5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.707635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a637339-2903-4973-8087-ad0a149ff5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.707907 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a637339-2903-4973-8087-ad0a149ff5b5 None admin] PATH_INFO: `/v3/projects/fbc389afc41e48b7b4bfa4d91dde83c0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.709035 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a637339-2903-4973-8087-ad0a149ff5b5 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc389afc41e48b7b4bfa4d91dde83c0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.710453 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a637339-2903-4973-8087-ad0a149ff5b5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.712310 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 448/904] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/projects/fbc389afc41e48b7b4bfa4d91dde83c0 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:09.718991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ab404-2558-43a3-a9af-91cebf29cd06 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.800833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.816561 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ab404-2558-43a3-a9af-91cebf29cd06 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.849950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bdac3-e22f-4b2a-b338-141e8e1f3856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.850538 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bdac3-e22f-4b2a-b338-141e8e1f3856 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.850812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bdac3-e22f-4b2a-b338-141e8e1f3856 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.851079 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bdac3-e22f-4b2a-b338-141e8e1f3856 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.851786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bdac3-e22f-4b2a-b338-141e8e1f3856 None admin] RBAC: Adding query filter params (role.id=81d287a8b606458d87c9607e8b48ac58, scope.project.id=fbc389afc41e48b7b4bfa4d91dde83c0, user.id=27de1ce6475948f0b2817b31b881ff9b) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:09.852090 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bdac3-e22f-4b2a-b338-141e8e1f3856 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.853529 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bdac3-e22f-4b2a-b338-141e8e1f3856 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.859261 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 457/905] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/role_assignments?user.id=27de1ce6475948f0b2817b31b881ff9b&scope.project.id=fbc389afc41e48b7b4bfa4d91dde83c0&role.id=81d287a8b606458d87c9607e8b48ac58 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:19.701843 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=48 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.856576 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=26 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:04.844133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef5d9c-21bb-49a8-b246-fe000bdf4b2e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.844133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef5d9c-21bb-49a8-b246-fe000bdf4b2e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.844133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef5d9c-21bb-49a8-b246-fe000bdf4b2e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.845535 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 449/906] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:17:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:04.852616 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e034bb-c7a8-463b-a5dc-260ceed97e28 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.853217 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e034bb-c7a8-463b-a5dc-260ceed97e28 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.853217 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e034bb-c7a8-463b-a5dc-260ceed97e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.877427 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a0e034bb-c7a8-463b-a5dc-260ceed97e28 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:04.878647 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a0e034bb-c7a8-463b-a5dc-260ceed97e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:04.971927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 458/907] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:05.271664 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef5d9c-21bb-49a8-b246-fe000bdf4b2e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:05.405878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef5d9c-21bb-49a8-b246-fe000bdf4b2e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.445166 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ec2ef-adcd-455a-acd2-192b6df43180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.446159 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ec2ef-adcd-455a-acd2-192b6df43180 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.446657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ec2ef-adcd-455a-acd2-192b6df43180 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.447120 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ec2ef-adcd-455a-acd2-192b6df43180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.464855 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ec2ef-adcd-455a-acd2-192b6df43180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:05.466653 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ec2ef-adcd-455a-acd2-192b6df43180 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7c7ec2ef-adcd-455a-acd2-192b6df43180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:05.481879 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:05.483452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:05.484429 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:05.484429 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:05.484429 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:17:05.485897 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 450/908] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:05.492571 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e034bb-c7a8-463b-a5dc-260ceed97e28 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:05.613806 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e034bb-c7a8-463b-a5dc-260ceed97e28 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.656757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:05.664307 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876a7aa-15f3-4358-914c-0844436ce563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.665027 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876a7aa-15f3-4358-914c-0844436ce563 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.665360 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876a7aa-15f3-4358-914c-0844436ce563 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.665593 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876a7aa-15f3-4358-914c-0844436ce563 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.666284 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876a7aa-15f3-4358-914c-0844436ce563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:05.666704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876a7aa-15f3-4358-914c-0844436ce563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:05.668094 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876a7aa-15f3-4358-914c-0844436ce563 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:05.684580 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 459/909] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:05.688380 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ec2ef-adcd-455a-acd2-192b6df43180 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:05.773167 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:05.830910 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ec2ef-adcd-455a-acd2-192b6df43180 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.872382 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562e582-9388-4efe-98b2-f14de4a2de58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.873552 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562e582-9388-4efe-98b2-f14de4a2de58 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.873765 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562e582-9388-4efe-98b2-f14de4a2de58 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.874039 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562e582-9388-4efe-98b2-f14de4a2de58 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.875846 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562e582-9388-4efe-98b2-f14de4a2de58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:05.877469 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562e582-9388-4efe-98b2-f14de4a2de58 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:05.903325 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 451/910] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Nov 2 07:17:05 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:08.157661 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9817c4-1821-4a14-950c-ed7617a53cfe None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.158197 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9817c4-1821-4a14-950c-ed7617a53cfe None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.158197 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9817c4-1821-4a14-950c-ed7617a53cfe None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.159570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 460/911] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:08.165198 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c55332-6395-40b8-bb30-e23679f2473e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.165921 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c55332-6395-40b8-bb30-e23679f2473e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.165921 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c55332-6395-40b8-bb30-e23679f2473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.185484 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-04c55332-6395-40b8-bb30-e23679f2473e None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:08.186823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-04c55332-6395-40b8-bb30-e23679f2473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:08.263858 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 452/912] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:08.537325 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9817c4-1821-4a14-950c-ed7617a53cfe None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.656186 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9817c4-1821-4a14-950c-ed7617a53cfe None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.699098 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14744ad-8cd2-40c8-8b1e-946474101feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.700827 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14744ad-8cd2-40c8-8b1e-946474101feb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.700827 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14744ad-8cd2-40c8-8b1e-946474101feb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.700827 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14744ad-8cd2-40c8-8b1e-946474101feb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.712894 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14744ad-8cd2-40c8-8b1e-946474101feb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d14744ad-8cd2-40c8-8b1e-946474101feb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:08.718320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:08.719309 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:08.720176 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 461/913] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:08.726118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c55332-6395-40b8-bb30-e23679f2473e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.771168 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:08.845713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c55332-6395-40b8-bb30-e23679f2473e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:08.885075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aafb2-6779-422d-8203-0fd79c6e9e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:08.885824 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aafb2-6779-422d-8203-0fd79c6e9e3b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:08.886079 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aafb2-6779-422d-8203-0fd79c6e9e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:08.886394 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aafb2-6779-422d-8203-0fd79c6e9e3b None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:08.887063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aafb2-6779-422d-8203-0fd79c6e9e3b None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:08.887368 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aafb2-6779-422d-8203-0fd79c6e9e3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:08.889418 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aafb2-6779-422d-8203-0fd79c6e9e3b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:08.898974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 453/914] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:08.905259 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14744ad-8cd2-40c8-8b1e-946474101feb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:08.965378 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.030155 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14744ad-8cd2-40c8-8b1e-946474101feb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.074340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deafbd3-be04-4269-a19f-d4a6f56f1654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.074980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deafbd3-be04-4269-a19f-d4a6f56f1654 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.075296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deafbd3-be04-4269-a19f-d4a6f56f1654 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.075524 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deafbd3-be04-4269-a19f-d4a6f56f1654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.087243 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deafbd3-be04-4269-a19f-d4a6f56f1654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.088911 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deafbd3-be04-4269-a19f-d4a6f56f1654 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5deafbd3-be04-4269-a19f-d4a6f56f1654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:09.105215 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:09.106372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:09.108024 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:09.108024 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.108024 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:09.108024 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 462/915] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:09.114840 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aafb2-6779-422d-8203-0fd79c6e9e3b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.224653 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.228385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aafb2-6779-422d-8203-0fd79c6e9e3b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.268134 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3a9c7-e93f-4acb-ae48-213be00cb4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.268825 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3a9c7-e93f-4acb-ae48-213be00cb4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.271277 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3a9c7-e93f-4acb-ae48-213be00cb4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.271277 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3a9c7-e93f-4acb-ae48-213be00cb4d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.271277 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3a9c7-e93f-4acb-ae48-213be00cb4d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:09.271277 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3a9c7-e93f-4acb-ae48-213be00cb4d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.271871 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3a9c7-e93f-4acb-ae48-213be00cb4d5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.284939 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 454/916] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:09.298007 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deafbd3-be04-4269-a19f-d4a6f56f1654 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.402089 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deafbd3-be04-4269-a19f-d4a6f56f1654 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.404591 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.441095 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0854b8-43c2-42b5-8321-f4f8ed8482bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.441692 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0854b8-43c2-42b5-8321-f4f8ed8482bd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.441964 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0854b8-43c2-42b5-8321-f4f8ed8482bd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.442303 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0854b8-43c2-42b5-8321-f4f8ed8482bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.450477 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0854b8-43c2-42b5-8321-f4f8ed8482bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.451913 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0854b8-43c2-42b5-8321-f4f8ed8482bd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4b0854b8-43c2-42b5-8321-f4f8ed8482bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:09.463828 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:09.464980 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:09.466010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:09.466010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:09.466010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:09.466010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:09.466010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:09.466010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:09.466010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:09.466010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:09.466010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:17:09.466010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:09.466010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 463/917] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:09.471624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3a9c7-e93f-4acb-ae48-213be00cb4d5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.595290 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3a9c7-e93f-4acb-ae48-213be00cb4d5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.631476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8e62c-51f2-41e4-8339-27d0b3c04ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.632090 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8e62c-51f2-41e4-8339-27d0b3c04ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.632366 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8e62c-51f2-41e4-8339-27d0b3c04ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.640574 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8e62c-51f2-41e4-8339-27d0b3c04ba8 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.640574 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8e62c-51f2-41e4-8339-27d0b3c04ba8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:09.640574 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8e62c-51f2-41e4-8339-27d0b3c04ba8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.640574 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8e62c-51f2-41e4-8339-27d0b3c04ba8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.664303 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 455/918] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:09.670674 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0854b8-43c2-42b5-8321-f4f8ed8482bd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.782291 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0854b8-43c2-42b5-8321-f4f8ed8482bd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:09.818553 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52e269-4db7-451f-839c-d39dfa592a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:09.819340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52e269-4db7-451f-839c-d39dfa592a97 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:09.819717 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52e269-4db7-451f-839c-d39dfa592a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:09.820085 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52e269-4db7-451f-839c-d39dfa592a97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:09.832551 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52e269-4db7-451f-839c-d39dfa592a97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:09.834160 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52e269-4db7-451f-839c-d39dfa592a97 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bb52e269-4db7-451f-839c-d39dfa592a97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:09.850886 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:09.852046 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:09.853246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:09.853246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:09.853246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:09.854374 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 464/919] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:09.862713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8e62c-51f2-41e4-8339-27d0b3c04ba8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:09.886731 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:09.995361 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8e62c-51f2-41e4-8339-27d0b3c04ba8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.033444 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed122bb-82fe-4b53-a6cb-b711cf2420a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.034091 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed122bb-82fe-4b53-a6cb-b711cf2420a2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.034371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed122bb-82fe-4b53-a6cb-b711cf2420a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.034649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed122bb-82fe-4b53-a6cb-b711cf2420a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.035340 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed122bb-82fe-4b53-a6cb-b711cf2420a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:10.035638 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed122bb-82fe-4b53-a6cb-b711cf2420a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.037081 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed122bb-82fe-4b53-a6cb-b711cf2420a2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.050141 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 456/920] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:10.055714 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52e269-4db7-451f-839c-d39dfa592a97 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.075220 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:10.175982 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52e269-4db7-451f-839c-d39dfa592a97 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.214360 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55393c8c-069b-4315-8344-873d0077a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.215152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55393c8c-069b-4315-8344-873d0077a33a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.215535 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55393c8c-069b-4315-8344-873d0077a33a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.215924 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55393c8c-069b-4315-8344-873d0077a33a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.224183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55393c8c-069b-4315-8344-873d0077a33a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.225909 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55393c8c-069b-4315-8344-873d0077a33a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-55393c8c-069b-4315-8344-873d0077a33a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:10.239282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:10.240442 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:10.241558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:10.241558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:10.241558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:10.241558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:10.241558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:10.241558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:10.241558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:10.241558 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:10.249106 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 465/921] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:10.250295 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed122bb-82fe-4b53-a6cb-b711cf2420a2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.348376 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:10.378321 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed122bb-82fe-4b53-a6cb-b711cf2420a2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.420594 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d83a14-aa34-4120-91f7-61de1fc8ecdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.421288 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d83a14-aa34-4120-91f7-61de1fc8ecdb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.421621 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d83a14-aa34-4120-91f7-61de1fc8ecdb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.421874 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d83a14-aa34-4120-91f7-61de1fc8ecdb None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.422684 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d83a14-aa34-4120-91f7-61de1fc8ecdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:10.423140 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d83a14-aa34-4120-91f7-61de1fc8ecdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.424808 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d83a14-aa34-4120-91f7-61de1fc8ecdb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.438072 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 457/922] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:10.444853 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55393c8c-069b-4315-8344-873d0077a33a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.532409 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:10.559711 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55393c8c-069b-4315-8344-873d0077a33a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.599615 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affff2d9-4b94-4931-81bb-96758763077e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.600437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affff2d9-4b94-4931-81bb-96758763077e None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.600860 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affff2d9-4b94-4931-81bb-96758763077e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.601213 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affff2d9-4b94-4931-81bb-96758763077e None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/19c9d282c1a74c25939c03e8dea6a345/roles/d276003aacd44f11a8b5e733a9a54f02` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.618438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affff2d9-4b94-4931-81bb-96758763077e None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=19c9d282c1a74c25939c03e8dea6a345, role_id=d276003aacd44f11a8b5e733a9a54f02)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.620064 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affff2d9-4b94-4931-81bb-96758763077e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.635367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 466/923] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:10 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/19c9d282c1a74c25939c03e8dea6a345/roles/d276003aacd44f11a8b5e733a9a54f02 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:17:12.933117 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dc8e1-d9f5-42f9-ad1c-f91b481318a1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:12.933686 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dc8e1-d9f5-42f9-ad1c-f91b481318a1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:12.933686 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dc8e1-d9f5-42f9-ad1c-f91b481318a1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:12.934910 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 458/924] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:12.940218 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509f68d-6b69-48c1-8721-03b5eab83f1a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:12.940733 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509f68d-6b69-48c1-8721-03b5eab83f1a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:12.940835 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509f68d-6b69-48c1-8721-03b5eab83f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:12.961604 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8509f68d-6b69-48c1-8721-03b5eab83f1a None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:12.962794 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8509f68d-6b69-48c1-8721-03b5eab83f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:13.035396 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 467/925] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:13.303338 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dc8e1-d9f5-42f9-ad1c-f91b481318a1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.421624 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dc8e1-d9f5-42f9-ad1c-f91b481318a1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:13.460070 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1041fe-5bee-4720-b6a8-f379b7fc38e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:13.460789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1041fe-5bee-4720-b6a8-f379b7fc38e6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.461402 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1041fe-5bee-4720-b6a8-f379b7fc38e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.461690 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1041fe-5bee-4720-b6a8-f379b7fc38e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.469845 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:13.474147 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1041fe-5bee-4720-b6a8-f379b7fc38e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-3b1041fe-5bee-4720-b6a8-f379b7fc38e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:13.479155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:13.480249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:17:13.481213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 459/926] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:13.487113 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509f68d-6b69-48c1-8721-03b5eab83f1a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.545369 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:13.592637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509f68d-6b69-48c1-8721-03b5eab83f1a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:13.632275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e723f-c8c4-4f81-b7f9-1157eb3dcbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:13.632990 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e723f-c8c4-4f81-b7f9-1157eb3dcbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.633354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e723f-c8c4-4f81-b7f9-1157eb3dcbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.633635 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e723f-c8c4-4f81-b7f9-1157eb3dcbf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.634297 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e723f-c8c4-4f81-b7f9-1157eb3dcbf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:13.635002 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e723f-c8c4-4f81-b7f9-1157eb3dcbf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:13.636072 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e723f-c8c4-4f81-b7f9-1157eb3dcbf3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:13.645023 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 468/927] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:13.651490 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1041fe-5bee-4720-b6a8-f379b7fc38e6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.772840 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1041fe-5bee-4720-b6a8-f379b7fc38e6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:13.811093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909131e1-280e-483b-b8cf-4904a5fb2d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:13.811709 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909131e1-280e-483b-b8cf-4904a5fb2d95 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.811987 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909131e1-280e-483b-b8cf-4904a5fb2d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.812257 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909131e1-280e-483b-b8cf-4904a5fb2d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.822548 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909131e1-280e-483b-b8cf-4904a5fb2d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:13.823982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909131e1-280e-483b-b8cf-4904a5fb2d95 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-909131e1-280e-483b-b8cf-4904a5fb2d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:13.839740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:13.840982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:13.842576 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:13.842576 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:13.842576 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:17:13.842576 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 460/928] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:13.847802 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e723f-c8c4-4f81-b7f9-1157eb3dcbf3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.967123 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e723f-c8c4-4f81-b7f9-1157eb3dcbf3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:13.986207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:14.004345 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe84189-3d94-47a0-8596-bf8c8535624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.004978 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe84189-3d94-47a0-8596-bf8c8535624b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.005255 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe84189-3d94-47a0-8596-bf8c8535624b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.005523 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe84189-3d94-47a0-8596-bf8c8535624b None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.006155 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe84189-3d94-47a0-8596-bf8c8535624b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:14.006452 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe84189-3d94-47a0-8596-bf8c8535624b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.007805 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe84189-3d94-47a0-8596-bf8c8535624b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.020031 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 469/929] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:14.026377 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909131e1-280e-483b-b8cf-4904a5fb2d95 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.080447 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:14.132184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909131e1-280e-483b-b8cf-4904a5fb2d95 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.171501 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74599b6-f431-4995-8ccc-33561c6d20d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.172137 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74599b6-f431-4995-8ccc-33561c6d20d4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.172415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74599b6-f431-4995-8ccc-33561c6d20d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.172727 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74599b6-f431-4995-8ccc-33561c6d20d4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.180205 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74599b6-f431-4995-8ccc-33561c6d20d4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.181836 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74599b6-f431-4995-8ccc-33561c6d20d4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b74599b6-f431-4995-8ccc-33561c6d20d4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:14.194789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:14.195975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:14.197149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:14.197149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:14.197149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:14.197149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:14.197149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:14.197149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:14.197149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:14.197149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:14.197149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:17:14.197149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:17:14.197149 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 461/930] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:14.202957 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe84189-3d94-47a0-8596-bf8c8535624b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.327356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe84189-3d94-47a0-8596-bf8c8535624b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.365701 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120075d0-1b1d-4a53-a2fe-f21904e44367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.366306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120075d0-1b1d-4a53-a2fe-f21904e44367 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.366582 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120075d0-1b1d-4a53-a2fe-f21904e44367 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.366851 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120075d0-1b1d-4a53-a2fe-f21904e44367 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.368147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120075d0-1b1d-4a53-a2fe-f21904e44367 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:14.368147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120075d0-1b1d-4a53-a2fe-f21904e44367 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.369352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120075d0-1b1d-4a53-a2fe-f21904e44367 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.389429 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 470/931] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:14.395826 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74599b6-f431-4995-8ccc-33561c6d20d4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.510591 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74599b6-f431-4995-8ccc-33561c6d20d4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.521581 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:14.546662 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26a087-4cf9-4591-bb2e-6efe78f7faeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.547333 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26a087-4cf9-4591-bb2e-6efe78f7faeb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.547610 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26a087-4cf9-4591-bb2e-6efe78f7faeb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.547919 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26a087-4cf9-4591-bb2e-6efe78f7faeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.623713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26a087-4cf9-4591-bb2e-6efe78f7faeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.625355 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26a087-4cf9-4591-bb2e-6efe78f7faeb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.729346 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 462/932] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:17:14.735847 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120075d0-1b1d-4a53-a2fe-f21904e44367 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.772715 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:14.846845 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120075d0-1b1d-4a53-a2fe-f21904e44367 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.886796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281065ed-fd1b-4a17-912a-cb4d08f0b941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.887431 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281065ed-fd1b-4a17-912a-cb4d08f0b941 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.887722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281065ed-fd1b-4a17-912a-cb4d08f0b941 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.888044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281065ed-fd1b-4a17-912a-cb4d08f0b941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.898693 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281065ed-fd1b-4a17-912a-cb4d08f0b941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.900238 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281065ed-fd1b-4a17-912a-cb4d08f0b941 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-281065ed-fd1b-4a17-912a-cb4d08f0b941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:14.918723 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:14.919883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:14.920993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:14.920993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:14.920993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:14.920993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 471/933] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:14.927126 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26a087-4cf9-4591-bb2e-6efe78f7faeb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.942462 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.042620 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26a087-4cf9-4591-bb2e-6efe78f7faeb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.080880 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342350f3-fbc7-4b17-ad7f-53c7e2ca4473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.081590 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342350f3-fbc7-4b17-ad7f-53c7e2ca4473 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.081889 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342350f3-fbc7-4b17-ad7f-53c7e2ca4473 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.082190 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342350f3-fbc7-4b17-ad7f-53c7e2ca4473 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.082982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342350f3-fbc7-4b17-ad7f-53c7e2ca4473 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:15.083282 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342350f3-fbc7-4b17-ad7f-53c7e2ca4473 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.084732 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342350f3-fbc7-4b17-ad7f-53c7e2ca4473 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.096379 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 463/934] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:15.102013 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281065ed-fd1b-4a17-912a-cb4d08f0b941 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.192680 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.208926 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281065ed-fd1b-4a17-912a-cb4d08f0b941 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.244174 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70619641-2a06-42bc-a927-787171a67152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.244948 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70619641-2a06-42bc-a927-787171a67152 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.245240 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70619641-2a06-42bc-a927-787171a67152 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.245488 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70619641-2a06-42bc-a927-787171a67152 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.253165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70619641-2a06-42bc-a927-787171a67152 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.254916 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70619641-2a06-42bc-a927-787171a67152 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-70619641-2a06-42bc-a927-787171a67152 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:15.266741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:15.267803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:15.268944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:15.268944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:15.268944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:15.268944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:15.268944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:15.268944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:15.268944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:15.268944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:15.268944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 472/935] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:15.273709 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342350f3-fbc7-4b17-ad7f-53c7e2ca4473 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.349981 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:15.382204 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342350f3-fbc7-4b17-ad7f-53c7e2ca4473 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.416744 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29425d36-1c03-4ce7-82de-06d7e79d5933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.417353 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425d36-1c03-4ce7-82de-06d7e79d5933 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.417631 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425d36-1c03-4ce7-82de-06d7e79d5933 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.417904 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425d36-1c03-4ce7-82de-06d7e79d5933 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.418609 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29425d36-1c03-4ce7-82de-06d7e79d5933 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:15.418882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29425d36-1c03-4ce7-82de-06d7e79d5933 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.420363 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29425d36-1c03-4ce7-82de-06d7e79d5933 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.434254 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 464/936] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:15.442012 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70619641-2a06-42bc-a927-787171a67152 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:15.554026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70619641-2a06-42bc-a927-787171a67152 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:15.588947 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025116da-cda1-4942-90ae-7f1190c664ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:15.589554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025116da-cda1-4942-90ae-7f1190c664ee None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:15.589833 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025116da-cda1-4942-90ae-7f1190c664ee None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:15.590132 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025116da-cda1-4942-90ae-7f1190c664ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:15.590852 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025116da-cda1-4942-90ae-7f1190c664ee None admin] RBAC: Adding query filter params (role.id=d276003aacd44f11a8b5e733a9a54f02, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=19c9d282c1a74c25939c03e8dea6a345) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:15.591675 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025116da-cda1-4942-90ae-7f1190c664ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:15.593129 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025116da-cda1-4942-90ae-7f1190c664ee None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:15.599067 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 473/937] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:15 2022] GET /identity/v3/role_assignments?user.id=19c9d282c1a74c25939c03e8dea6a345&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=d276003aacd44f11a8b5e733a9a54f02 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:17.992774 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68375377-af23-43c2-a5dc-b012a3260c00 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.993284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68375377-af23-43c2-a5dc-b012a3260c00 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.993284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68375377-af23-43c2-a5dc-b012a3260c00 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.994584 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 465/938] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:18.000407 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44f363-497b-459c-b5db-dd06e167df9d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.000884 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44f363-497b-459c-b5db-dd06e167df9d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.000996 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44f363-497b-459c-b5db-dd06e167df9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.020645 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8d44f363-497b-459c-b5db-dd06e167df9d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:18.021849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8d44f363-497b-459c-b5db-dd06e167df9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:18.024605 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:18.096400 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 474/939] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:18.368635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68375377-af23-43c2-a5dc-b012a3260c00 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.486693 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68375377-af23-43c2-a5dc-b012a3260c00 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.521353 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:18.530672 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682b549-047a-48bb-85c1-5beeee5412a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.531332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682b549-047a-48bb-85c1-5beeee5412a1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.531592 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682b549-047a-48bb-85c1-5beeee5412a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.531866 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682b549-047a-48bb-85c1-5beeee5412a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.542568 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682b549-047a-48bb-85c1-5beeee5412a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8682b549-047a-48bb-85c1-5beeee5412a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:18.547882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:18.549067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:17:18.550158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 466/940] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:18.555186 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44f363-497b-459c-b5db-dd06e167df9d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.678793 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44f363-497b-459c-b5db-dd06e167df9d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.724974 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631759a8-9990-49d1-82d3-1e97fa1d1c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.725594 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631759a8-9990-49d1-82d3-1e97fa1d1c2a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.725902 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631759a8-9990-49d1-82d3-1e97fa1d1c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.726152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631759a8-9990-49d1-82d3-1e97fa1d1c2a None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.726812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631759a8-9990-49d1-82d3-1e97fa1d1c2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:18.727115 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631759a8-9990-49d1-82d3-1e97fa1d1c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:18.728472 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631759a8-9990-49d1-82d3-1e97fa1d1c2a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:18.737220 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 475/941] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:18.742975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682b549-047a-48bb-85c1-5beeee5412a1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.846418 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682b549-047a-48bb-85c1-5beeee5412a1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:18.886450 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f0aa8-7a25-4b26-934f-683ae1eb7fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:18.887127 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f0aa8-7a25-4b26-934f-683ae1eb7fbe None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:18.887414 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f0aa8-7a25-4b26-934f-683ae1eb7fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:18.887689 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f0aa8-7a25-4b26-934f-683ae1eb7fbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:18.901467 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f0aa8-7a25-4b26-934f-683ae1eb7fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:18.903063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f0aa8-7a25-4b26-934f-683ae1eb7fbe None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0e7f0aa8-7a25-4b26-934f-683ae1eb7fbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:18.922040 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:18.923016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:18.923903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:18.923903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:18.923903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:17:18.923903 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 467/942] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:18.930302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631759a8-9990-49d1-82d3-1e97fa1d1c2a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:18.933350 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.037267 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631759a8-9990-49d1-82d3-1e97fa1d1c2a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.076153 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ba1fb-eb89-4280-9727-233774e0b0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.076809 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ba1fb-eb89-4280-9727-233774e0b0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.077089 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ba1fb-eb89-4280-9727-233774e0b0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.077362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ba1fb-eb89-4280-9727-233774e0b0a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.078011 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ba1fb-eb89-4280-9727-233774e0b0a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:19.078340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ba1fb-eb89-4280-9727-233774e0b0a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.079680 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ba1fb-eb89-4280-9727-233774e0b0a8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.092681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 476/943] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:19.098964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f0aa8-7a25-4b26-934f-683ae1eb7fbe None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.144081 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.205187 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f0aa8-7a25-4b26-934f-683ae1eb7fbe None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.242092 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205495f-9e47-4f2e-a9b3-cf16ba7baf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.242862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205495f-9e47-4f2e-a9b3-cf16ba7baf05 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.243146 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205495f-9e47-4f2e-a9b3-cf16ba7baf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.243428 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205495f-9e47-4f2e-a9b3-cf16ba7baf05 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.252652 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205495f-9e47-4f2e-a9b3-cf16ba7baf05 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.254084 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205495f-9e47-4f2e-a9b3-cf16ba7baf05 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7205495f-9e47-4f2e-a9b3-cf16ba7baf05 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:19.266578 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:19.267862 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:19.269018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:19.269018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:19.269018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:19.269018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:19.269018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:19.269018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:19.269018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:19.269018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:19.269018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:17:19.269018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:17:19.269018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 468/944] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:19.274235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ba1fb-eb89-4280-9727-233774e0b0a8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.345856 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.387712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ba1fb-eb89-4280-9727-233774e0b0a8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.424190 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc0fdc-169a-41e0-b851-e38c0114deb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.424924 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0fdc-169a-41e0-b851-e38c0114deb9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.425207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0fdc-169a-41e0-b851-e38c0114deb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.425507 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0fdc-169a-41e0-b851-e38c0114deb9 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.426396 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc0fdc-169a-41e0-b851-e38c0114deb9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:19.426704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc0fdc-169a-41e0-b851-e38c0114deb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.428468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc0fdc-169a-41e0-b851-e38c0114deb9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.447079 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 477/945] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:19.452905 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205495f-9e47-4f2e-a9b3-cf16ba7baf05 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.565983 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205495f-9e47-4f2e-a9b3-cf16ba7baf05 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.567447 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.603046 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561c303-1d60-4fc4-a834-88774cc65c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.603675 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561c303-1d60-4fc4-a834-88774cc65c77 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.603955 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561c303-1d60-4fc4-a834-88774cc65c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.604276 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561c303-1d60-4fc4-a834-88774cc65c77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.615893 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561c303-1d60-4fc4-a834-88774cc65c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.617366 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561c303-1d60-4fc4-a834-88774cc65c77 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-3561c303-1d60-4fc4-a834-88774cc65c77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:19.632800 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:19.639786 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:19.640805 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:19.640805 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:19.640805 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:17:19.640805 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 469/946] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:19.641985 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc0fdc-169a-41e0-b851-e38c0114deb9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.761638 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc0fdc-169a-41e0-b851-e38c0114deb9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.784865 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:19.801064 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3c6b1-8a5d-4920-9ca6-574e2852f990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.801686 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3c6b1-8a5d-4920-9ca6-574e2852f990 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.801987 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3c6b1-8a5d-4920-9ca6-574e2852f990 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.802236 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3c6b1-8a5d-4920-9ca6-574e2852f990 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.802853 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3c6b1-8a5d-4920-9ca6-574e2852f990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:19.803159 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3c6b1-8a5d-4920-9ca6-574e2852f990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:19.804508 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3c6b1-8a5d-4920-9ca6-574e2852f990 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:19.816230 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 478/947] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:19.822184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561c303-1d60-4fc4-a834-88774cc65c77 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:19.943695 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561c303-1d60-4fc4-a834-88774cc65c77 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:19.989307 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6a1af-7a2b-46f1-868c-46a3353198f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:19.990120 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6a1af-7a2b-46f1-868c-46a3353198f0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:19.990396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6a1af-7a2b-46f1-868c-46a3353198f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:19.990819 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6a1af-7a2b-46f1-868c-46a3353198f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:19.999902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6a1af-7a2b-46f1-868c-46a3353198f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:20.001364 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6a1af-7a2b-46f1-868c-46a3353198f0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7ea6a1af-7a2b-46f1-868c-46a3353198f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:20.015996 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:20.017184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:20.018267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:20.018267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:20.018267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:20.018267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:20.018267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:20.018267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:20.018267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:20.018267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:17:20.018267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 470/948] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:20.023530 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3c6b1-8a5d-4920-9ca6-574e2852f990 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:20.135302 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3c6b1-8a5d-4920-9ca6-574e2852f990 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:20.174757 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d00546-8b8d-4312-ba58-71ea399705a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:20.175397 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d00546-8b8d-4312-ba58-71ea399705a6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:20.175671 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d00546-8b8d-4312-ba58-71ea399705a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:20.175940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d00546-8b8d-4312-ba58-71ea399705a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:20.176642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d00546-8b8d-4312-ba58-71ea399705a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:20.176946 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d00546-8b8d-4312-ba58-71ea399705a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:20.178367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d00546-8b8d-4312-ba58-71ea399705a6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:20.191620 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 479/949] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:20.197743 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6a1af-7a2b-46f1-868c-46a3353198f0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:20.217943 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:20.310061 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6a1af-7a2b-46f1-868c-46a3353198f0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:20.352763 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb429d34-b9fc-4351-99a6-7b59b6e61651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:20.353723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb429d34-b9fc-4351-99a6-7b59b6e61651 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:20.353723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb429d34-b9fc-4351-99a6-7b59b6e61651 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:20.354242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb429d34-b9fc-4351-99a6-7b59b6e61651 None admin] PATH_INFO: `/v3/projects/41551cd452f845849527d7c06ff6e719/users/19c9d282c1a74c25939c03e8dea6a345/roles/81d287a8b606458d87c9607e8b48ac58` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:20.356631 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb429d34-b9fc-4351-99a6-7b59b6e61651 None admin] RBAC: Authorizing `identity:create_grant(project_id=41551cd452f845849527d7c06ff6e719, user_id=19c9d282c1a74c25939c03e8dea6a345, role_id=81d287a8b606458d87c9607e8b48ac58)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:20.358464 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb429d34-b9fc-4351-99a6-7b59b6e61651 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:20.376601 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 471/950] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:20 2022] PUT /identity/v3/projects/41551cd452f845849527d7c06ff6e719/users/19c9d282c1a74c25939c03e8dea6a345/roles/81d287a8b606458d87c9607e8b48ac58 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:17:22.701881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b2fb4-7bc5-46f0-8564-46680fcfbee4 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.702411 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b2fb4-7bc5-46f0-8564-46680fcfbee4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.702411 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b2fb4-7bc5-46f0-8564-46680fcfbee4 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.704185 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 480/951] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:22.710844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f9eb1-462d-4a28-8828-3e812a822290 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:22.711593 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f9eb1-462d-4a28-8828-3e812a822290 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:22.711716 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f9eb1-462d-4a28-8828-3e812a822290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:22.734247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-074f9eb1-462d-4a28-8828-3e812a822290 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:22.735441 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-074f9eb1-462d-4a28-8828-3e812a822290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:22.810486 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 472/952] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:23.089181 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b2fb4-7bc5-46f0-8564-46680fcfbee4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.134773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:23.196912 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b2fb4-7bc5-46f0-8564-46680fcfbee4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.230975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ee85a-73e6-454f-8e8b-0fa386352ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.232091 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee85a-73e6-454f-8e8b-0fa386352ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.232368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee85a-73e6-454f-8e8b-0fa386352ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.232671 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee85a-73e6-454f-8e8b-0fa386352ab9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.243047 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ee85a-73e6-454f-8e8b-0fa386352ab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ce3ee85a-73e6-454f-8e8b-0fa386352ab9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:23.248428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:23.249589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:23.253695 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 481/953] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:23.255991 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f9eb1-462d-4a28-8828-3e812a822290 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.259632 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:23.374629 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f9eb1-462d-4a28-8828-3e812a822290 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.412249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7b3aa-a130-45da-97f9-492bfe76439b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.412899 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7b3aa-a130-45da-97f9-492bfe76439b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.413184 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7b3aa-a130-45da-97f9-492bfe76439b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.413459 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7b3aa-a130-45da-97f9-492bfe76439b None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.414123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7b3aa-a130-45da-97f9-492bfe76439b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:23.414434 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7b3aa-a130-45da-97f9-492bfe76439b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.415802 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7b3aa-a130-45da-97f9-492bfe76439b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.425191 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 473/954] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:23.431301 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ee85a-73e6-454f-8e8b-0fa386352ab9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.538133 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:23.541262 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ee85a-73e6-454f-8e8b-0fa386352ab9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.579038 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f715d4-1693-4cf3-b57e-591f7556a27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.579733 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f715d4-1693-4cf3-b57e-591f7556a27c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.580042 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f715d4-1693-4cf3-b57e-591f7556a27c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.580349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f715d4-1693-4cf3-b57e-591f7556a27c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.591499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f715d4-1693-4cf3-b57e-591f7556a27c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.593229 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f715d4-1693-4cf3-b57e-591f7556a27c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-34f715d4-1693-4cf3-b57e-591f7556a27c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:23.609390 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:23.610554 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:23.611437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:23.611437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:23.611437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:23.611437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 482/955] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:17:23.616741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7b3aa-a130-45da-97f9-492bfe76439b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.688032 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:23.727964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7b3aa-a130-45da-97f9-492bfe76439b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.767660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43481bbe-b83b-4323-8a5f-0a28f829df80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.768366 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43481bbe-b83b-4323-8a5f-0a28f829df80 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.768677 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43481bbe-b83b-4323-8a5f-0a28f829df80 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.768966 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43481bbe-b83b-4323-8a5f-0a28f829df80 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.769590 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43481bbe-b83b-4323-8a5f-0a28f829df80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:23.769890 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43481bbe-b83b-4323-8a5f-0a28f829df80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.771249 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43481bbe-b83b-4323-8a5f-0a28f829df80 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.785559 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 474/956] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:23.793177 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f715d4-1693-4cf3-b57e-591f7556a27c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:23.909125 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f715d4-1693-4cf3-b57e-591f7556a27c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:23.948774 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785590eb-64ee-40d8-aaef-85ecabf4c44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:23.949406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785590eb-64ee-40d8-aaef-85ecabf4c44f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:23.949684 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785590eb-64ee-40d8-aaef-85ecabf4c44f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:23.949972 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785590eb-64ee-40d8-aaef-85ecabf4c44f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:23.959030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785590eb-64ee-40d8-aaef-85ecabf4c44f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:23.960843 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785590eb-64ee-40d8-aaef-85ecabf4c44f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-785590eb-64ee-40d8-aaef-85ecabf4c44f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:17:23.973950 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:23.974991 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:23.977468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:23.977468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:23.977468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:17:23.977468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:17:23.977468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:17:23.977468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:17:23.977468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:17:23.977468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:17:23.977468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:17:23.977468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:23.977468 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 483/957] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:23.981991 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43481bbe-b83b-4323-8a5f-0a28f829df80 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:24.087420 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43481bbe-b83b-4323-8a5f-0a28f829df80 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:24.115876 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:24.125144 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1edef0-16e5-46b3-b15e-3cfe97d68854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:24.125761 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1edef0-16e5-46b3-b15e-3cfe97d68854 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:24.126042 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1edef0-16e5-46b3-b15e-3cfe97d68854 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:24.126313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1edef0-16e5-46b3-b15e-3cfe97d68854 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:24.127056 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1edef0-16e5-46b3-b15e-3cfe97d68854 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:24.127363 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1edef0-16e5-46b3-b15e-3cfe97d68854 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:24.128837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1edef0-16e5-46b3-b15e-3cfe97d68854 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:24.148715 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 475/958] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Nov 2 07:17:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:24.155043 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785590eb-64ee-40d8-aaef-85ecabf4c44f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:24.171112 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:24.259713 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785590eb-64ee-40d8-aaef-85ecabf4c44f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:24.294626 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af63f3-4ba1-4f26-b346-32a9608fce8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:24.295255 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af63f3-4ba1-4f26-b346-32a9608fce8f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:24.295533 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af63f3-4ba1-4f26-b346-32a9608fce8f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:24.295913 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af63f3-4ba1-4f26-b346-32a9608fce8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:24.386993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af63f3-4ba1-4f26-b346-32a9608fce8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:24.388455 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af63f3-4ba1-4f26-b346-32a9608fce8f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:24.402007 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:24.487722 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 484/959] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 07:17:24.493674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1edef0-16e5-46b3-b15e-3cfe97d68854 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:24.597751 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1edef0-16e5-46b3-b15e-3cfe97d68854 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:24.631075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2c30d-2626-448b-bef3-36968c984825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:24.631710 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2c30d-2626-448b-bef3-36968c984825 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:24.631986 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2c30d-2626-448b-bef3-36968c984825 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:24.632253 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2c30d-2626-448b-bef3-36968c984825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:24.643017 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2c30d-2626-448b-bef3-36968c984825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:24.644457 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2c30d-2626-448b-bef3-36968c984825 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a5c2c30d-2626-448b-bef3-36968c984825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:17:24.658595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:24.659657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:17:24.660708 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:17:24.660708 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:17:24.660708 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:17:24.660708 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 476/960] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:24.665342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af63f3-4ba1-4f26-b346-32a9608fce8f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:24.759464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af63f3-4ba1-4f26-b346-32a9608fce8f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:24.772356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:24.792471 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2d3eb-beb5-452c-8347-e8495578a0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:24.793085 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d3eb-beb5-452c-8347-e8495578a0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:24.793360 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d3eb-beb5-452c-8347-e8495578a0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:24.794416 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d3eb-beb5-452c-8347-e8495578a0a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:24.794771 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2d3eb-beb5-452c-8347-e8495578a0a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:24.795066 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2d3eb-beb5-452c-8347-e8495578a0a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:24.796401 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2d3eb-beb5-452c-8347-e8495578a0a5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:24.807464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 485/961] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Nov 2 07:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:24.813655 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2c30d-2626-448b-bef3-36968c984825 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:24.847635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:24.912507 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2c30d-2626-448b-bef3-36968c984825 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:24.944238 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0560b-e1ce-4eb5-9b8b-090119cbc40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:24.944882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0560b-e1ce-4eb5-9b8b-090119cbc40a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:24.945163 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0560b-e1ce-4eb5-9b8b-090119cbc40a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:24.945435 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0560b-e1ce-4eb5-9b8b-090119cbc40a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:24.952356 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0560b-e1ce-4eb5-9b8b-090119cbc40a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:24.953809 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0560b-e1ce-4eb5-9b8b-090119cbc40a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0ea0560b-e1ce-4eb5-9b8b-090119cbc40a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:17:24.964957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:24.966558 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:24.967649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:17:24.967649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:17:24.967649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:17:24.967649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:17:24.967649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:17:24.967649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:17:24.967649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:17:24.967649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Nov 02 07:17:24.967649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 477/962] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Nov 2 07:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:24.972567 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2d3eb-beb5-452c-8347-e8495578a0a5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:25.068012 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2d3eb-beb5-452c-8347-e8495578a0a5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:25.100773 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09d1db-efb9-412d-93c7-cc170ef365d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:25.101376 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09d1db-efb9-412d-93c7-cc170ef365d3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:25.101659 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09d1db-efb9-412d-93c7-cc170ef365d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:25.101933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09d1db-efb9-412d-93c7-cc170ef365d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:25.102620 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09d1db-efb9-412d-93c7-cc170ef365d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:25.102929 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09d1db-efb9-412d-93c7-cc170ef365d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:25.104350 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09d1db-efb9-412d-93c7-cc170ef365d3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:25.115918 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 486/963] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Nov 2 07:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:25.121743 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0560b-e1ce-4eb5-9b8b-090119cbc40a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:25.213333 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:25.219399 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0560b-e1ce-4eb5-9b8b-090119cbc40a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:25.251685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d317cb2-e4a9-4479-b1d3-daa0bf1d11f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:25.252271 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d317cb2-e4a9-4479-b1d3-daa0bf1d11f6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:25.252569 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d317cb2-e4a9-4479-b1d3-daa0bf1d11f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:25.252844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d317cb2-e4a9-4479-b1d3-daa0bf1d11f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:25.253557 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d317cb2-e4a9-4479-b1d3-daa0bf1d11f6 None admin] RBAC: Adding query filter params (role.id=81d287a8b606458d87c9607e8b48ac58, scope.project.id=41551cd452f845849527d7c06ff6e719, user.id=19c9d282c1a74c25939c03e8dea6a345) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:25.253861 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d317cb2-e4a9-4479-b1d3-daa0bf1d11f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:25.255268 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d317cb2-e4a9-4479-b1d3-daa0bf1d11f6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:25.260977 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 478/964] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:25 2022] GET /identity/v3/role_assignments?user.id=19c9d282c1a74c25939c03e8dea6a345&scope.project.id=41551cd452f845849527d7c06ff6e719&role.id=81d287a8b606458d87c9607e8b48ac58 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:27.401483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7d3f1-f17d-4ded-9fda-1772527140b0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.402041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7d3f1-f17d-4ded-9fda-1772527140b0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.402041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7d3f1-f17d-4ded-9fda-1772527140b0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.403403 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 487/965] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:27.408692 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27474173-aa3a-4963-8da8-18afd2264e22 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.409092 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27474173-aa3a-4963-8da8-18afd2264e22 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.409239 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27474173-aa3a-4963-8da8-18afd2264e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.427604 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-27474173-aa3a-4963-8da8-18afd2264e22 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:27.428812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-27474173-aa3a-4963-8da8-18afd2264e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:27.492348 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 479/966] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:27.763235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7d3f1-f17d-4ded-9fda-1772527140b0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:27.793455 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:27.862191 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7d3f1-f17d-4ded-9fda-1772527140b0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:27.894315 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8b037-0499-4a24-a74a-baccad3264b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:27.894927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8b037-0499-4a24-a74a-baccad3264b7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:27.895201 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8b037-0499-4a24-a74a-baccad3264b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:27.895474 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8b037-0499-4a24-a74a-baccad3264b7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:27.896042 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8b037-0499-4a24-a74a-baccad3264b7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:27.897406 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8b037-0499-4a24-a74a-baccad3264b7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9ba8b037-0499-4a24-a74a-baccad3264b7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:27.905938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:27.907246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:27.908331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 488/967] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Nov 2 07:17:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:27.913016 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27474173-aa3a-4963-8da8-18afd2264e22 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:28.006211 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:28.015668 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27474173-aa3a-4963-8da8-18afd2264e22 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:28.047988 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f17f89-696f-48b2-bbf2-b1dee93f640d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:28.048606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f17f89-696f-48b2-bbf2-b1dee93f640d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:28.048883 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f17f89-696f-48b2-bbf2-b1dee93f640d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:28.049152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f17f89-696f-48b2-bbf2-b1dee93f640d None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:28.049766 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f17f89-696f-48b2-bbf2-b1dee93f640d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:28.050062 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f17f89-696f-48b2-bbf2-b1dee93f640d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:28.051407 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f17f89-696f-48b2-bbf2-b1dee93f640d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:28.055977 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 480/968] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Nov 2 07:17:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:28.062607 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8b037-0499-4a24-a74a-baccad3264b7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:28.152409 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:28.159059 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8b037-0499-4a24-a74a-baccad3264b7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:28.190803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63261828-b030-49a6-9a46-9a092c8bc190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:28.191387 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63261828-b030-49a6-9a46-9a092c8bc190 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:28.191663 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63261828-b030-49a6-9a46-9a092c8bc190 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:28.191935 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63261828-b030-49a6-9a46-9a092c8bc190 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:28.192564 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63261828-b030-49a6-9a46-9a092c8bc190 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:28.192869 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63261828-b030-49a6-9a46-9a092c8bc190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:28.194214 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63261828-b030-49a6-9a46-9a092c8bc190 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:28.198429 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 489/969] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Nov 2 07:17:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:30.301436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b961-dbbd-4ccd-b030-b6dc01a1a3f6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:30.302027 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b961-dbbd-4ccd-b030-b6dc01a1a3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:30.302027 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b961-dbbd-4ccd-b030-b6dc01a1a3f6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:30.303483 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 481/970] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:30.309345 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168e4c6-d0c9-4aa2-9624-ac148adf7903 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:30.309798 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168e4c6-d0c9-4aa2-9624-ac148adf7903 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:30.309970 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168e4c6-d0c9-4aa2-9624-ac148adf7903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:30.331975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2168e4c6-d0c9-4aa2-9624-ac148adf7903 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:30.333547 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2168e4c6-d0c9-4aa2-9624-ac148adf7903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:30.399362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 490/971] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:30.663831 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227b961-dbbd-4ccd-b030-b6dc01a1a3f6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:30.762495 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227b961-dbbd-4ccd-b030-b6dc01a1a3f6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:30.794842 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0981a6c-8784-405a-8f9e-1bbbcdf752ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:30.795436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0981a6c-8784-405a-8f9e-1bbbcdf752ec None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:30.795714 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0981a6c-8784-405a-8f9e-1bbbcdf752ec None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:30.795984 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0981a6c-8784-405a-8f9e-1bbbcdf752ec None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:30.796654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0981a6c-8784-405a-8f9e-1bbbcdf752ec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:30.798002 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0981a6c-8784-405a-8f9e-1bbbcdf752ec None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:30.811616 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 482/972] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Nov 2 07:17:30 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:32.940565 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5238ea5-a220-4dc0-b380-791f45b037ec None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:32.941078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5238ea5-a220-4dc0-b380-791f45b037ec None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:32.941078 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5238ea5-a220-4dc0-b380-791f45b037ec None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:32.942379 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 491/973] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:32.947490 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01107fc8-a22e-49a0-a42e-6d075dc324b0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:32.947906 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01107fc8-a22e-49a0-a42e-6d075dc324b0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:32.948047 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01107fc8-a22e-49a0-a42e-6d075dc324b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:32.966532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-01107fc8-a22e-49a0-a42e-6d075dc324b0 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:32.967716 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-01107fc8-a22e-49a0-a42e-6d075dc324b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:32.976791 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:33.040211 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 483/974] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:17:33.308967 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5238ea5-a220-4dc0-b380-791f45b037ec None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:33.407783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:33.418117 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5238ea5-a220-4dc0-b380-791f45b037ec None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:33.451087 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c94832-419a-415b-8d3c-66dbeb534323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:33.451701 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c94832-419a-415b-8d3c-66dbeb534323 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:33.451975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c94832-419a-415b-8d3c-66dbeb534323 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:33.452244 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c94832-419a-415b-8d3c-66dbeb534323 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:33.452848 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c94832-419a-415b-8d3c-66dbeb534323 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:33.454199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c94832-419a-415b-8d3c-66dbeb534323 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-66c94832-419a-415b-8d3c-66dbeb534323 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:33.462877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:33.464139 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:33.465425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 492/975] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Nov 2 07:17:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:33.470033 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01107fc8-a22e-49a0-a42e-6d075dc324b0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:33.568179 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01107fc8-a22e-49a0-a42e-6d075dc324b0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:33.603183 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6abaab-6a3b-4e7b-aaba-db85107c84a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:33.603784 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6abaab-6a3b-4e7b-aaba-db85107c84a1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:33.604066 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6abaab-6a3b-4e7b-aaba-db85107c84a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:33.604340 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6abaab-6a3b-4e7b-aaba-db85107c84a1 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:33.604989 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6abaab-6a3b-4e7b-aaba-db85107c84a1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:33.605294 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6abaab-6a3b-4e7b-aaba-db85107c84a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:33.606696 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6abaab-6a3b-4e7b-aaba-db85107c84a1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:33.611616 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 484/976] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Nov 2 07:17:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:33.617617 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c94832-419a-415b-8d3c-66dbeb534323 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:33.716874 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c94832-419a-415b-8d3c-66dbeb534323 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:33.749677 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9f523-b454-4816-8b54-b2764dd67ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:33.750272 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9f523-b454-4816-8b54-b2764dd67ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:33.750552 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9f523-b454-4816-8b54-b2764dd67ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:33.750827 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9f523-b454-4816-8b54-b2764dd67ba6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:33.751516 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9f523-b454-4816-8b54-b2764dd67ba6 None admin] RBAC: Adding query filter params (interface=public, service_id=bc2fcf01a78d4fa0a384585033322a98, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:33.751824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9f523-b454-4816-8b54-b2764dd67ba6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:33.753204 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9f523-b454-4816-8b54-b2764dd67ba6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:33.758007 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 493/977] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Nov 2 07:17:33 2022] GET /identity/v3/endpoints?service_id=bc2fcf01a78d4fa0a384585033322a98&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:33.763634 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6abaab-6a3b-4e7b-aaba-db85107c84a1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:33.767358 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:33.860126 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6abaab-6a3b-4e7b-aaba-db85107c84a1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:33.893067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d385b-9e5c-4dc0-b6f9-b99592e1e4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:33.893667 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d385b-9e5c-4dc0-b6f9-b99592e1e4dc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:33.893950 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d385b-9e5c-4dc0-b6f9-b99592e1e4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:33.894223 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d385b-9e5c-4dc0-b6f9-b99592e1e4dc None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:33.894817 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d385b-9e5c-4dc0-b6f9-b99592e1e4dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:33.896165 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d385b-9e5c-4dc0-b6f9-b99592e1e4dc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:33.902763 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 485/978] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Nov 2 07:17:33 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:17:36.015359 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e20bc-1194-465b-9853-d7b414064f1b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.015868 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e20bc-1194-465b-9853-d7b414064f1b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.015868 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e20bc-1194-465b-9853-d7b414064f1b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.017199 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 494/979] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:36.022276 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4c0f7-9d2d-458e-9747-b6f54bc2dd72 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.022692 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4c0f7-9d2d-458e-9747-b6f54bc2dd72 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.022831 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4c0f7-9d2d-458e-9747-b6f54bc2dd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.041936 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bdd4c0f7-9d2d-458e-9747-b6f54bc2dd72 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:36.043131 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bdd4c0f7-9d2d-458e-9747-b6f54bc2dd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:36.102606 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:36.108169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 486/980] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:36.377339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e20bc-1194-465b-9853-d7b414064f1b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:36.396190 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:36.475546 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e20bc-1194-465b-9853-d7b414064f1b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:36.507799 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01aed7-a303-4c32-8399-b80f8e725f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:36.508412 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01aed7-a303-4c32-8399-b80f8e725f16 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.508719 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01aed7-a303-4c32-8399-b80f8e725f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.508993 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01aed7-a303-4c32-8399-b80f8e725f16 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.509568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01aed7-a303-4c32-8399-b80f8e725f16 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:36.510919 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01aed7-a303-4c32-8399-b80f8e725f16 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2c01aed7-a303-4c32-8399-b80f8e725f16 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:36.519519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:36.520712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Nov 02 07:17:36.521735 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 495/981] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Nov 2 07:17:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:36.526518 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4c0f7-9d2d-458e-9747-b6f54bc2dd72 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:36.621170 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4c0f7-9d2d-458e-9747-b6f54bc2dd72 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:36.653922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416a34b-26f5-4407-889b-787ccf2a7442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:36.654520 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416a34b-26f5-4407-889b-787ccf2a7442 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.654793 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416a34b-26f5-4407-889b-787ccf2a7442 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.655056 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416a34b-26f5-4407-889b-787ccf2a7442 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.655670 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416a34b-26f5-4407-889b-787ccf2a7442 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:36.655966 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416a34b-26f5-4407-889b-787ccf2a7442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:36.657345 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416a34b-26f5-4407-889b-787ccf2a7442 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:36.662270 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 487/982] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Nov 2 07:17:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:36.667944 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01aed7-a303-4c32-8399-b80f8e725f16 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:36.747644 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:36.765881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01aed7-a303-4c32-8399-b80f8e725f16 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:36.798160 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ad213-d716-4ec0-a6cc-111a2fb8dff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:36.798748 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ad213-d716-4ec0-a6cc-111a2fb8dff6 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:36.799023 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ad213-d716-4ec0-a6cc-111a2fb8dff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:36.799290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ad213-d716-4ec0-a6cc-111a2fb8dff6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:36.799932 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ad213-d716-4ec0-a6cc-111a2fb8dff6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:36.801289 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ad213-d716-4ec0-a6cc-111a2fb8dff6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:36.819227 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 496/983] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Nov 2 07:17:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:46.659330 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=69 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:46.815141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:40.004701 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef02eb-6d59-46e4-a6a5-0ba9e61ee70f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:40.005167 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef02eb-6d59-46e4-a6a5-0ba9e61ee70f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:40.005167 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef02eb-6d59-46e4-a6a5-0ba9e61ee70f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:40.012321 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddafd9-66af-424a-b11f-87f947b90a63 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:40.013459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddafd9-66af-424a-b11f-87f947b90a63 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:40.014242 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddafd9-66af-424a-b11f-87f947b90a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:40.016899 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 488/984] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:18:40 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:40.194044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-82ddafd9-66af-424a-b11f-87f947b90a63 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:40.195280 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-82ddafd9-66af-424a-b11f-87f947b90a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:40.399372 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 497/985] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:18:40.693960 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491d745-e5c5-46b0-a4e1-8d1b174e6c65 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:40.694586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491d745-e5c5-46b0-a4e1-8d1b174e6c65 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:40.695067 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491d745-e5c5-46b0-a4e1-8d1b174e6c65 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:40.697648 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 489/986] 10.222.0.15 () {54 vars in 965 bytes} [Wed Nov 2 07:18:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:40.703183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b247-0f00-4fdb-b51b-e75f57c1f8f5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:40.703464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b247-0f00-4fdb-b51b-e75f57c1f8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:40.703740 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b247-0f00-4fdb-b51b-e75f57c1f8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:40.792290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1f42b247-0f00-4fdb-b51b-e75f57c1f8f5 None None] MFA Rules not processed for user `36450acd551f40d08a4bc21c9a9c8877`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:40.793733 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1f42b247-0f00-4fdb-b51b-e75f57c1f8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:40.820531 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:40.890949 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 498/987] 10.222.0.15 () {58 vars in 1062 bytes} [Wed Nov 2 07:18:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:18:40.900473 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491d745-e5c5-46b0-a4e1-8d1b174e6c65 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:40.902334 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5491d745-e5c5-46b0-a4e1-8d1b174e6c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:41.043951 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491d745-e5c5-46b0-a4e1-8d1b174e6c65 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:41.138351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf125b04-e80d-47e5-9bbe-07e9e5bbff7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:41.138973 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf125b04-e80d-47e5-9bbe-07e9e5bbff7d service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:41.139252 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf125b04-e80d-47e5-9bbe-07e9e5bbff7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:41.139550 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf125b04-e80d-47e5-9bbe-07e9e5bbff7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:41.141657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bf125b04-e80d-47e5-9bbe-07e9e5bbff7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:41.206844 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:41.268205 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf125b04-e80d-47e5-9bbe-07e9e5bbff7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:41.269692 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf125b04-e80d-47e5-9bbe-07e9e5bbff7d service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:41.289901 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 490/988] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:18:43.795339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18510d0c-df9d-4629-92f0-7c542385f6ba None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:43.795889 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18510d0c-df9d-4629-92f0-7c542385f6ba None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:43.795985 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18510d0c-df9d-4629-92f0-7c542385f6ba None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:43.797313 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 499/989] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:43.802762 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8e144-c565-438d-aded-d69ff73470df None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:43.803241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8e144-c565-438d-aded-d69ff73470df None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:43.803332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8e144-c565-438d-aded-d69ff73470df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:43.876812 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e6f8e144-c565-438d-aded-d69ff73470df None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:43.878046 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e6f8e144-c565-438d-aded-d69ff73470df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:43.974289 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 491/990] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:18:44.253969 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18510d0c-df9d-4629-92f0-7c542385f6ba None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:44.255505 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-18510d0c-df9d-4629-92f0-7c542385f6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:44.358296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18510d0c-df9d-4629-92f0-7c542385f6ba None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:44.428211 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:44.430884 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c66fe1-1e2a-4051-95db-43397aae401b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:44.431681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c66fe1-1e2a-4051-95db-43397aae401b service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:44.432001 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c66fe1-1e2a-4051-95db-43397aae401b service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:44.432466 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c66fe1-1e2a-4051-95db-43397aae401b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:44.434659 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-77c66fe1-1e2a-4051-95db-43397aae401b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:44.542440 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c66fe1-1e2a-4051-95db-43397aae401b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:44.543781 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c66fe1-1e2a-4051-95db-43397aae401b service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:44.561567 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 500/991] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:18:47.006168 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a685b-82c2-4f97-a3f8-92ac48a9caa0 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:47.006669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a685b-82c2-4f97-a3f8-92ac48a9caa0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:47.006669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a685b-82c2-4f97-a3f8-92ac48a9caa0 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:47.007967 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 492/992] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:47.013363 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df031a7d-aa58-40c9-80ae-b8c920c31311 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:47.013738 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df031a7d-aa58-40c9-80ae-b8c920c31311 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:47.013908 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df031a7d-aa58-40c9-80ae-b8c920c31311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:47.078161 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-df031a7d-aa58-40c9-80ae-b8c920c31311 None None] MFA Rules not processed for user `a0147318766a4812b77983481386e463`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:47.079366 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-df031a7d-aa58-40c9-80ae-b8c920c31311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:47.131347 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:47.175475 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 501/993] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:47 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:18:47.457870 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a685b-82c2-4f97-a3f8-92ac48a9caa0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:47.459393 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f53a685b-82c2-4f97-a3f8-92ac48a9caa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:47.469994 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:47.562091 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a685b-82c2-4f97-a3f8-92ac48a9caa0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:47.633493 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fac21-0df6-41b6-8421-3a3fd1123956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:47.634633 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fac21-0df6-41b6-8421-3a3fd1123956 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:47.634912 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fac21-0df6-41b6-8421-3a3fd1123956 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:47.635188 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fac21-0df6-41b6-8421-3a3fd1123956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:47.636805 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-570fac21-0df6-41b6-8421-3a3fd1123956 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:47.698186 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:47.743052 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570fac21-0df6-41b6-8421-3a3fd1123956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:47.744957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570fac21-0df6-41b6-8421-3a3fd1123956 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:47.762615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 493/994] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:18:47 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:18:50.432307 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fef9d-5857-4b45-a9ac-05ae5f180d6a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:50.432929 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fef9d-5857-4b45-a9ac-05ae5f180d6a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:50.432929 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fef9d-5857-4b45-a9ac-05ae5f180d6a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:50.434241 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 502/995] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:50.439489 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4cb60-0ea9-4ccc-81a4-d6fbc28e8e67 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:50.439899 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4cb60-0ea9-4ccc-81a4-d6fbc28e8e67 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:50.440053 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4cb60-0ea9-4ccc-81a4-d6fbc28e8e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:50.505774 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1de4cb60-0ea9-4ccc-81a4-d6fbc28e8e67 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:50.506974 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1de4cb60-0ea9-4ccc-81a4-d6fbc28e8e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:50.600882 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 494/996] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:18:50.885923 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fef9d-5857-4b45-a9ac-05ae5f180d6a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:50.887475 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7d7fef9d-5857-4b45-a9ac-05ae5f180d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:50.990082 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fef9d-5857-4b45-a9ac-05ae5f180d6a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:51.060771 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab78d00-5552-43cb-8134-aa694c13c1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:51.061363 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab78d00-5552-43cb-8134-aa694c13c1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:51.061636 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab78d00-5552-43cb-8134-aa694c13c1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:51.061904 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab78d00-5552-43cb-8134-aa694c13c1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:51.063506 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-aab78d00-5552-43cb-8134-aa694c13c1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:51.070189 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:51.171490 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab78d00-5552-43cb-8134-aa694c13c1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:51.172849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab78d00-5552-43cb-8134-aa694c13c1b0 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:51.190832 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 503/997] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:18:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:18:53.968761 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad53dfb-7ebc-4345-ac91-69c0391fe2e3 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:53.969271 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad53dfb-7ebc-4345-ac91-69c0391fe2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:53.969271 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad53dfb-7ebc-4345-ac91-69c0391fe2e3 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:53.970555 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 495/998] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:18:53.975737 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df7f00-225e-411e-b803-e74619f32c4f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:53.976127 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df7f00-225e-411e-b803-e74619f32c4f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:53.976284 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df7f00-225e-411e-b803-e74619f32c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:54.049809 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e5df7f00-225e-411e-b803-e74619f32c4f None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:54.051067 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e5df7f00-225e-411e-b803-e74619f32c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:54.113401 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:54.158811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 504/999] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:18:54.446957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad53dfb-7ebc-4345-ac91-69c0391fe2e3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:54.448516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3ad53dfb-7ebc-4345-ac91-69c0391fe2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:54.468626 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:54.552345 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad53dfb-7ebc-4345-ac91-69c0391fe2e3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:54.625863 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfa12f-e31a-4984-b52f-07cb41b03810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:54.626478 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfa12f-e31a-4984-b52f-07cb41b03810 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:54.626757 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfa12f-e31a-4984-b52f-07cb41b03810 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:54.627030 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfa12f-e31a-4984-b52f-07cb41b03810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:54.628651 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-94dfa12f-e31a-4984-b52f-07cb41b03810 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:54.704998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:54.744110 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfa12f-e31a-4984-b52f-07cb41b03810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:54.745581 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfa12f-e31a-4984-b52f-07cb41b03810 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:54.763083 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 496/1000] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:18:57.063570 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1991c-970f-4576-b734-5697c22d0b1d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:57.064082 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1991c-970f-4576-b734-5697c22d0b1d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:57.064178 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1991c-970f-4576-b734-5697c22d0b1d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:57.065575 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 505/1001] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:57.071817 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c46bb-b166-4a89-8df5-1af4b6dc3e7d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:57.072284 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c46bb-b166-4a89-8df5-1af4b6dc3e7d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:57.072396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c46bb-b166-4a89-8df5-1af4b6dc3e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:57.140788 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-978c46bb-b166-4a89-8df5-1af4b6dc3e7d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:57.141975 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-978c46bb-b166-4a89-8df5-1af4b6dc3e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:57.240833 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 497/1002] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:18:57.517229 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1991c-970f-4576-b734-5697c22d0b1d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:57.518768 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a4d1991c-970f-4576-b734-5697c22d0b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:57.623114 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1991c-970f-4576-b734-5697c22d0b1d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:57.698472 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d628569-bef6-41cc-adf5-ff02000bcf80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:57.699186 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d628569-bef6-41cc-adf5-ff02000bcf80 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:57.699466 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d628569-bef6-41cc-adf5-ff02000bcf80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:57.699741 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d628569-bef6-41cc-adf5-ff02000bcf80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:57.701381 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4d628569-bef6-41cc-adf5-ff02000bcf80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:57.718906 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:57.812401 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d628569-bef6-41cc-adf5-ff02000bcf80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:57.813750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d628569-bef6-41cc-adf5-ff02000bcf80 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:57.832459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 506/1003] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:00.038989 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1ba79-d3c2-4a06-b507-3637fdb30323 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:00.039476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1ba79-d3c2-4a06-b507-3637fdb30323 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:00.040140 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1ba79-d3c2-4a06-b507-3637fdb30323 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:00.041583 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 498/1004] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:00.046796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb2a9b-7123-4f15-b66d-51db1ddfd676 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:00.047237 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb2a9b-7123-4f15-b66d-51db1ddfd676 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:00.047354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb2a9b-7123-4f15-b66d-51db1ddfd676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:00.113440 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-87eb2a9b-7123-4f15-b66d-51db1ddfd676 None None] MFA Rules not processed for user `a0147318766a4812b77983481386e463`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:00.114643 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-87eb2a9b-7123-4f15-b66d-51db1ddfd676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:00.188368 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:00.224353 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 507/1005] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:00 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:00.506872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1ba79-d3c2-4a06-b507-3637fdb30323 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:00.508480 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8ea1ba79-d3c2-4a06-b507-3637fdb30323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:00.545521 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:00.618396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1ba79-d3c2-4a06-b507-3637fdb30323 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:00.693195 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df197cc4-f678-4759-b9b4-7e1f40bb1440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:00.693808 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197cc4-f678-4759-b9b4-7e1f40bb1440 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:00.694087 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197cc4-f678-4759-b9b4-7e1f40bb1440 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:00.694362 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197cc4-f678-4759-b9b4-7e1f40bb1440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:00.695982 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-df197cc4-f678-4759-b9b4-7e1f40bb1440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:00.774116 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:00.805863 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df197cc4-f678-4759-b9b4-7e1f40bb1440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:00.807207 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df197cc4-f678-4759-b9b4-7e1f40bb1440 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:00.826699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 499/1006] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:03.196355 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507ad67-955f-458e-973c-30c875ebbb8d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:03.196943 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507ad67-955f-458e-973c-30c875ebbb8d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:03.196943 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507ad67-955f-458e-973c-30c875ebbb8d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:03.198286 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 508/1007] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:03.203542 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0e336-1d6d-417f-8e19-fe4f79d95aba None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:03.203952 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0e336-1d6d-417f-8e19-fe4f79d95aba None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:03.204091 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0e336-1d6d-417f-8e19-fe4f79d95aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:03.268438 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bce0e336-1d6d-417f-8e19-fe4f79d95aba None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:03.269651 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bce0e336-1d6d-417f-8e19-fe4f79d95aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:03.365197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 500/1008] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:03.642127 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507ad67-955f-458e-973c-30c875ebbb8d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:03.643747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6507ad67-955f-458e-973c-30c875ebbb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:03.744052 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507ad67-955f-458e-973c-30c875ebbb8d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:03.814425 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b0f83-f388-4211-bfb0-9a9a0d4fb2ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:03.815018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b0f83-f388-4211-bfb0-9a9a0d4fb2ab service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:03.815290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b0f83-f388-4211-bfb0-9a9a0d4fb2ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:03.815555 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b0f83-f388-4211-bfb0-9a9a0d4fb2ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:03.817165 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cf6b0f83-f388-4211-bfb0-9a9a0d4fb2ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:03.843599 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:03.925270 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b0f83-f388-4211-bfb0-9a9a0d4fb2ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:03.926628 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b0f83-f388-4211-bfb0-9a9a0d4fb2ab service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:03.945198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 509/1009] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:06.120872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4d11f-03e1-4a6f-88ed-f23079a31604 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:06.121396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4d11f-03e1-4a6f-88ed-f23079a31604 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:06.121396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4d11f-03e1-4a6f-88ed-f23079a31604 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:06.122721 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 501/1010] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:06.127857 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060ae78-2044-4557-8935-798c21bda7fb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:06.128266 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060ae78-2044-4557-8935-798c21bda7fb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:06.128403 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060ae78-2044-4557-8935-798c21bda7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:06.192948 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c060ae78-2044-4557-8935-798c21bda7fb None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:06.194156 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c060ae78-2044-4557-8935-798c21bda7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:06.269329 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:06.292327 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 510/1011] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:06.575121 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4d11f-03e1-4a6f-88ed-f23079a31604 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:06.576676 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d9d4d11f-03e1-4a6f-88ed-f23079a31604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:06.616196 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:06.681145 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4d11f-03e1-4a6f-88ed-f23079a31604 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:06.753842 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd26fb9-0333-4fd5-9543-88389b74f65e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:06.754436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd26fb9-0333-4fd5-9543-88389b74f65e service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:06.754707 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd26fb9-0333-4fd5-9543-88389b74f65e service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:06.755003 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd26fb9-0333-4fd5-9543-88389b74f65e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:06.756595 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bdd26fb9-0333-4fd5-9543-88389b74f65e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:06.841310 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:06.867020 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd26fb9-0333-4fd5-9543-88389b74f65e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:06.869060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd26fb9-0333-4fd5-9543-88389b74f65e service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:06.886634 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 502/1012] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:19:09.311877 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974d756-4668-42d4-a90f-c9cbe5c0c524 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:09.312394 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974d756-4668-42d4-a90f-c9cbe5c0c524 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:09.312485 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974d756-4668-42d4-a90f-c9cbe5c0c524 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:09.313892 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 511/1013] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:09.319708 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81da40e-3fcb-4099-b624-17dac5d0106d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:09.320243 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81da40e-3fcb-4099-b624-17dac5d0106d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:09.320243 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81da40e-3fcb-4099-b624-17dac5d0106d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:09.384954 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a81da40e-3fcb-4099-b624-17dac5d0106d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:09.386147 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a81da40e-3fcb-4099-b624-17dac5d0106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:09.482719 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 503/1014] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:09.762861 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974d756-4668-42d4-a90f-c9cbe5c0c524 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:09.764399 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4974d756-4668-42d4-a90f-c9cbe5c0c524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:09.865546 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974d756-4668-42d4-a90f-c9cbe5c0c524 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:09.938405 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34914987-f79c-49dc-ae7d-0932a519606e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:09.939011 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34914987-f79c-49dc-ae7d-0932a519606e service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:09.939290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34914987-f79c-49dc-ae7d-0932a519606e service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:09.939562 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34914987-f79c-49dc-ae7d-0932a519606e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:09.941164 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-34914987-f79c-49dc-ae7d-0932a519606e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:09.976326 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:10.048777 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34914987-f79c-49dc-ae7d-0932a519606e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:10.050125 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34914987-f79c-49dc-ae7d-0932a519606e service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:10.067940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 512/1015] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:14.132666 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac6410-786b-4a34-9737-2a8a6decd96c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:14.133308 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac6410-786b-4a34-9737-2a8a6decd96c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:14.133308 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac6410-786b-4a34-9737-2a8a6decd96c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:14.134604 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 504/1016] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:14.139808 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cad959-6631-44c9-9b05-afdcd5da6e76 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:14.140246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cad959-6631-44c9-9b05-afdcd5da6e76 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:14.140377 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cad959-6631-44c9-9b05-afdcd5da6e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:14.205780 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-07cad959-6631-44c9-9b05-afdcd5da6e76 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:14.206908 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-07cad959-6631-44c9-9b05-afdcd5da6e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:14.294186 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:14.306573 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 513/1017] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:14.590738 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac6410-786b-4a34-9737-2a8a6decd96c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:14.592291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2dac6410-786b-4a34-9737-2a8a6decd96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:14.641960 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:14.697680 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac6410-786b-4a34-9737-2a8a6decd96c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:14.770147 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde9efb-2642-4436-8cfb-ba4d140c7c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:14.770751 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde9efb-2642-4436-8cfb-ba4d140c7c4c service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:14.771029 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde9efb-2642-4436-8cfb-ba4d140c7c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:14.771302 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde9efb-2642-4436-8cfb-ba4d140c7c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:14.772912 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ecde9efb-2642-4436-8cfb-ba4d140c7c4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:14.868791 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:14.885615 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde9efb-2642-4436-8cfb-ba4d140c7c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:14.887070 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde9efb-2642-4436-8cfb-ba4d140c7c4c service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:14.905657 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 505/1018] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:17.697206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e684613-94ce-4d46-ad45-6fda794e561d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:17.697750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e684613-94ce-4d46-ad45-6fda794e561d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:17.697750 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e684613-94ce-4d46-ad45-6fda794e561d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:17.699143 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 514/1019] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:17.704357 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a57be-add8-466b-a9ac-d05554f4d704 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:17.704789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a57be-add8-466b-a9ac-d05554f4d704 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:17.704940 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a57be-add8-466b-a9ac-d05554f4d704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:17.770841 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-083a57be-add8-466b-a9ac-d05554f4d704 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:17.772037 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-083a57be-add8-466b-a9ac-d05554f4d704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:17.870403 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 506/1020] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:18.166605 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fabf61-6db9-4bf6-8325-0ef89b4f9533 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:18.166887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fabf61-6db9-4bf6-8325-0ef89b4f9533 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:18.167161 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fabf61-6db9-4bf6-8325-0ef89b4f9533 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:18.168512 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 515/1021] 10.222.0.15 () {54 vars in 965 bytes} [Wed Nov 2 07:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:18.174402 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1458bd5-989e-4ca4-bc15-e7a7cdb849d9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:18.174712 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1458bd5-989e-4ca4-bc15-e7a7cdb849d9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:18.174984 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1458bd5-989e-4ca4-bc15-e7a7cdb849d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:18.250329 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:18.259806 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c1458bd5-989e-4ca4-bc15-e7a7cdb849d9 None None] MFA Rules not processed for user `36450acd551f40d08a4bc21c9a9c8877`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:18.261033 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c1458bd5-989e-4ca4-bc15-e7a7cdb849d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:18.339868 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 507/1022] 10.222.0.15 () {58 vars in 1062 bytes} [Wed Nov 2 07:19:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:18.349918 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fabf61-6db9-4bf6-8325-0ef89b4f9533 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:18.351521 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-14fabf61-6db9-4bf6-8325-0ef89b4f9533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:18.457933 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fabf61-6db9-4bf6-8325-0ef89b4f9533 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:18.533119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bdd73-4750-4f0e-90c2-4c33985360dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:18.533720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bdd73-4750-4f0e-90c2-4c33985360dc service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:18.533994 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bdd73-4750-4f0e-90c2-4c33985360dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:18.534258 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bdd73-4750-4f0e-90c2-4c33985360dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:18.535836 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-726bdd73-4750-4f0e-90c2-4c33985360dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:18.582312 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:18.674871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bdd73-4750-4f0e-90c2-4c33985360dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:18.676313 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bdd73-4750-4f0e-90c2-4c33985360dc service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:18.697520 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 516/1023] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:23.553101 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeb65f-e9f7-4e41-9c56-4318dc8db3e3 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:23.553564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeb65f-e9f7-4e41-9c56-4318dc8db3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:23.553682 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeb65f-e9f7-4e41-9c56-4318dc8db3e3 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:23.555042 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 508/1024] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:23.560477 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6b54e-efa6-4ced-9e2c-3bc1cfd12837 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:23.560927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6b54e-efa6-4ced-9e2c-3bc1cfd12837 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:23.561051 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6b54e-efa6-4ced-9e2c-3bc1cfd12837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:23.630131 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7eb6b54e-efa6-4ced-9e2c-3bc1cfd12837 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:23.631333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7eb6b54e-efa6-4ced-9e2c-3bc1cfd12837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:23.727394 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:23.729581 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 517/1025] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:24.009641 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aeb65f-e9f7-4e41-9c56-4318dc8db3e3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:24.011253 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-51aeb65f-e9f7-4e41-9c56-4318dc8db3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:24.117890 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aeb65f-e9f7-4e41-9c56-4318dc8db3e3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:24.166583 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:24.191866 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed368b0-b312-448a-b407-57b2942319ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:24.192471 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed368b0-b312-448a-b407-57b2942319ae service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:24.192791 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed368b0-b312-448a-b407-57b2942319ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:24.193057 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed368b0-b312-448a-b407-57b2942319ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:24.194711 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bed368b0-b312-448a-b407-57b2942319ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:24.301769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed368b0-b312-448a-b407-57b2942319ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:24.303128 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed368b0-b312-448a-b407-57b2942319ae service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:24.322695 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 509/1026] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:29.238560 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d15a93-d4a9-4ba6-b7ef-061411599588 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:29.239026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d15a93-d4a9-4ba6-b7ef-061411599588 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:29.239109 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d15a93-d4a9-4ba6-b7ef-061411599588 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:29.240571 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 518/1027] 10.222.0.15 () {54 vars in 921 bytes} [Wed Nov 2 07:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:29.258171 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00911d-ee05-4556-8999-231f89e01617 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:29.258171 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00911d-ee05-4556-8999-231f89e01617 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:29.258171 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00911d-ee05-4556-8999-231f89e01617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:29.390286 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1d00911d-ee05-4556-8999-231f89e01617 None None] MFA Rules not processed for user `19c9d282c1a74c25939c03e8dea6a345`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:29.391921 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1d00911d-ee05-4556-8999-231f89e01617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:29.411696 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:29.517159 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 510/1028] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Nov 2 07:19:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:33.685975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcdb84-bfcd-4bf6-93db-9127f421b7b6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:33.686561 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcdb84-bfcd-4bf6-93db-9127f421b7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:33.686561 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcdb84-bfcd-4bf6-93db-9127f421b7b6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:33.688037 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 519/1029] 10.222.0.15 () {54 vars in 921 bytes} [Wed Nov 2 07:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:33.693155 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcea3a-7296-4237-8980-452595762075 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:33.693966 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcea3a-7296-4237-8980-452595762075 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:33.695271 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcea3a-7296-4237-8980-452595762075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:33.782436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a9fcea3a-7296-4237-8980-452595762075 None None] MFA Rules not processed for user `19c9d282c1a74c25939c03e8dea6a345`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:33.783803 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a9fcea3a-7296-4237-8980-452595762075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:33.862104 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:33.908207 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 511/1030] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Nov 2 07:19:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:35.186552 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982790f9-0477-44f1-bf5d-402ee1db906f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:35.187109 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982790f9-0477-44f1-bf5d-402ee1db906f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:35.187109 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982790f9-0477-44f1-bf5d-402ee1db906f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:35.188420 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 520/1031] 10.222.0.15 () {54 vars in 921 bytes} [Wed Nov 2 07:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:35.193713 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45038c12-0236-44a7-b825-66a5ed3e2986 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:35.194131 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45038c12-0236-44a7-b825-66a5ed3e2986 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:35.194258 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45038c12-0236-44a7-b825-66a5ed3e2986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:35.270107 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-45038c12-0236-44a7-b825-66a5ed3e2986 None None] MFA Rules not processed for user `19c9d282c1a74c25939c03e8dea6a345`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:35.271312 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-45038c12-0236-44a7-b825-66a5ed3e2986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:35.378704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 512/1032] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Nov 2 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:38.838542 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a378ba3-c406-442b-ba4d-dd004083027b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:38.839428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a378ba3-c406-442b-ba4d-dd004083027b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:38.839812 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a378ba3-c406-442b-ba4d-dd004083027b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:38.841289 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 521/1033] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:38.846635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07734df3-1f27-4187-a4a7-70d79d8090d0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:38.847020 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07734df3-1f27-4187-a4a7-70d79d8090d0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:38.847178 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07734df3-1f27-4187-a4a7-70d79d8090d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:38.884729 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:38.920384 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-07734df3-1f27-4187-a4a7-70d79d8090d0 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:38.921629 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-07734df3-1f27-4187-a4a7-70d79d8090d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:39.024246 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 513/1034] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:19:39.320264 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e56c5-3492-4704-b110-ba61912918a2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:39.320793 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e56c5-3492-4704-b110-ba61912918a2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:39.321206 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e56c5-3492-4704-b110-ba61912918a2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:39.322937 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 522/1035] 10.222.0.15 () {54 vars in 961 bytes} [Wed Nov 2 07:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:39.333072 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ed8d2-0127-4d8d-97cf-438694755fd2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:39.333609 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ed8d2-0127-4d8d-97cf-438694755fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:39.334521 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ed8d2-0127-4d8d-97cf-438694755fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:39.437145 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-962ed8d2-0127-4d8d-97cf-438694755fd2 None None] MFA Rules not processed for user `00ab674b018e474ab363e8e17d225e27`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:39.438482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-962ed8d2-0127-4d8d-97cf-438694755fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:39.491195 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:39.566283 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 514/1036] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Nov 2 07:19:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:39.573918 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e56c5-3492-4704-b110-ba61912918a2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:39.575577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-287e56c5-3492-4704-b110-ba61912918a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:39.731814 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e56c5-3492-4704-b110-ba61912918a2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:39.809474 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:39.850979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b74b0-d417-42eb-99ed-9ee9f1785973 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:39.851612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b74b0-d417-42eb-99ed-9ee9f1785973 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:39.851851 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b74b0-d417-42eb-99ed-9ee9f1785973 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:39.852120 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b74b0-d417-42eb-99ed-9ee9f1785973 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:39.853715 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a78b74b0-d417-42eb-99ed-9ee9f1785973 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:39.955450 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b74b0-d417-42eb-99ed-9ee9f1785973 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:39.957067 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b74b0-d417-42eb-99ed-9ee9f1785973 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:39.976605 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 523/1037] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:42.338144 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30ca90-eff2-49d8-a8ff-e62d6c998d9d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:42.338738 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30ca90-eff2-49d8-a8ff-e62d6c998d9d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:42.338738 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30ca90-eff2-49d8-a8ff-e62d6c998d9d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:42.340037 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 515/1038] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:42.352700 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca87a3c-ffaf-4687-baae-dc563f7b5421 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:42.353153 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca87a3c-ffaf-4687-baae-dc563f7b5421 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:42.353801 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca87a3c-ffaf-4687-baae-dc563f7b5421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:42.454721 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-aca87a3c-ffaf-4687-baae-dc563f7b5421 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:42.455977 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-aca87a3c-ffaf-4687-baae-dc563f7b5421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:42.498031 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:42.560919 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 524/1039] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:42.855158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30ca90-eff2-49d8-a8ff-e62d6c998d9d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:42.856798 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cf30ca90-eff2-49d8-a8ff-e62d6c998d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:43.048131 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30ca90-eff2-49d8-a8ff-e62d6c998d9d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:43.068343 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:43.183817 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016f85f-f83d-48a3-b4b2-ffb8b59cb3a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:43.184434 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016f85f-f83d-48a3-b4b2-ffb8b59cb3a9 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:43.184748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016f85f-f83d-48a3-b4b2-ffb8b59cb3a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:43.185015 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016f85f-f83d-48a3-b4b2-ffb8b59cb3a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:43.186607 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2016f85f-f83d-48a3-b4b2-ffb8b59cb3a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:43.278255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016f85f-f83d-48a3-b4b2-ffb8b59cb3a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:43.279757 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016f85f-f83d-48a3-b4b2-ffb8b59cb3a9 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:43.299368 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 516/1040] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:45.774596 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b21ff-b17e-4bcc-a5f3-fc05592350e9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:45.775126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b21ff-b17e-4bcc-a5f3-fc05592350e9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:45.775126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b21ff-b17e-4bcc-a5f3-fc05592350e9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:45.776657 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 525/1041] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:45.781683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c9a0e-2452-4875-a1ca-b109a210c383 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:45.782120 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c9a0e-2452-4875-a1ca-b109a210c383 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:45.782215 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c9a0e-2452-4875-a1ca-b109a210c383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:45.798807 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:45.854371 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-102c9a0e-2452-4875-a1ca-b109a210c383 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:45.855482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-102c9a0e-2452-4875-a1ca-b109a210c383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:45.972869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 517/1042] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:46.254920 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f324e87-1ba1-48c0-80ab-61cfe0e0322a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:46.255217 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f324e87-1ba1-48c0-80ab-61cfe0e0322a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:46.255493 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f324e87-1ba1-48c0-80ab-61cfe0e0322a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:46.257606 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 526/1043] 10.222.0.15 () {54 vars in 961 bytes} [Wed Nov 2 07:19:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:46.262712 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51d3cb-22c5-43ec-b742-8d0e96984e69 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:46.263083 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51d3cb-22c5-43ec-b742-8d0e96984e69 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:46.263464 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51d3cb-22c5-43ec-b742-8d0e96984e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:46.345803 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bf51d3cb-22c5-43ec-b742-8d0e96984e69 None None] MFA Rules not processed for user `00ab674b018e474ab363e8e17d225e27`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:46.347112 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bf51d3cb-22c5-43ec-b742-8d0e96984e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:46.384215 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:46.476530 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 518/1044] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Nov 2 07:19:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:19:46.484480 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f324e87-1ba1-48c0-80ab-61cfe0e0322a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:46.486141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3f324e87-1ba1-48c0-80ab-61cfe0e0322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:46.646918 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f324e87-1ba1-48c0-80ab-61cfe0e0322a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:46.669479 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:46.770749 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5544f-4a14-438f-aacc-1b8a37069b58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:46.771362 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5544f-4a14-438f-aacc-1b8a37069b58 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:46.771640 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5544f-4a14-438f-aacc-1b8a37069b58 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:46.771934 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5544f-4a14-438f-aacc-1b8a37069b58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:46.773589 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f2f5544f-4a14-438f-aacc-1b8a37069b58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:46.875929 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5544f-4a14-438f-aacc-1b8a37069b58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:46.877552 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5544f-4a14-438f-aacc-1b8a37069b58 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:46.896566 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 527/1045] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:46.903126 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ab1ff-8cd3-422d-8b40-eebfb7e71339 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:46.903544 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ab1ff-8cd3-422d-8b40-eebfb7e71339 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:46.903642 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ab1ff-8cd3-422d-8b40-eebfb7e71339 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:46.905017 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 519/1046] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:46.910269 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337c54d-59b1-4b47-b211-0f2d46f0b3cd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:46.910551 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337c54d-59b1-4b47-b211-0f2d46f0b3cd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:46.910826 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337c54d-59b1-4b47-b211-0f2d46f0b3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:46.944502 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:46.982769 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f337c54d-59b1-4b47-b211-0f2d46f0b3cd None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:46.984117 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f337c54d-59b1-4b47-b211-0f2d46f0b3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:47.108356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 528/1047] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:47.436896 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5b68c-6dbc-4777-88a9-737ece505767 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:47.436896 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5b68c-6dbc-4777-88a9-737ece505767 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:47.436896 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5b68c-6dbc-4777-88a9-737ece505767 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:47.438421 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 520/1048] 10.222.0.15 () {54 vars in 963 bytes} [Wed Nov 2 07:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:47.443436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8297d-ba95-49ac-b9d0-2540fa4f8196 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:47.443720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8297d-ba95-49ac-b9d0-2540fa4f8196 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:47.444623 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8297d-ba95-49ac-b9d0-2540fa4f8196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:47.524005 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c6d8297d-ba95-49ac-b9d0-2540fa4f8196 None None] MFA Rules not processed for user `e0f9d9dc1f7143e0be709e17d1a5f484`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:47.525235 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c6d8297d-ba95-49ac-b9d0-2540fa4f8196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:47.573057 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:47.605234 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 529/1049] 10.222.0.15 () {58 vars in 1060 bytes} [Wed Nov 2 07:19:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:47.612748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5b68c-6dbc-4777-88a9-737ece505767 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:47.614240 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-34d5b68c-6dbc-4777-88a9-737ece505767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:47.729691 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5b68c-6dbc-4777-88a9-737ece505767 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:47.771557 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:47.807815 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fab81e-6a29-4a07-bdc0-3e53af1d65cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0f9d9dc1f7143e0be709e17d1a5f484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:47.808420 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fab81e-6a29-4a07-bdc0-3e53af1d65cf service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:47.808760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fab81e-6a29-4a07-bdc0-3e53af1d65cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:47.809092 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fab81e-6a29-4a07-bdc0-3e53af1d65cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:47.810674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e9fab81e-6a29-4a07-bdc0-3e53af1d65cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:47.946732 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fab81e-6a29-4a07-bdc0-3e53af1d65cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:47.949271 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fab81e-6a29-4a07-bdc0-3e53af1d65cf service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:47.978876 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 521/1050] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:49.125792 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4003e05-a97c-49d1-b12a-171fc296c5c7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:49.126303 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4003e05-a97c-49d1-b12a-171fc296c5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:49.126303 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4003e05-a97c-49d1-b12a-171fc296c5c7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:49.127639 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 530/1051] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:49.133685 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159213-756e-4eda-8888-139919157ec7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:49.134116 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159213-756e-4eda-8888-139919157ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:49.134272 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159213-756e-4eda-8888-139919157ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:49.199322 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0b159213-756e-4eda-8888-139919157ec7 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:49.200522 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0b159213-756e-4eda-8888-139919157ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:49.221796 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:49.299351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 522/1052] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:49.594936 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4003e05-a97c-49d1-b12a-171fc296c5c7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:49.596455 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c4003e05-a97c-49d1-b12a-171fc296c5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:49.753547 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4003e05-a97c-49d1-b12a-171fc296c5c7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:49.807275 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:49.872294 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae954ac-e066-4bf5-9b09-7864dd1e6ab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:49.872940 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae954ac-e066-4bf5-9b09-7864dd1e6ab5 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:49.873212 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae954ac-e066-4bf5-9b09-7864dd1e6ab5 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:49.873484 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae954ac-e066-4bf5-9b09-7864dd1e6ab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:49.875058 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fae954ac-e066-4bf5-9b09-7864dd1e6ab5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:49.975981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae954ac-e066-4bf5-9b09-7864dd1e6ab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:49.977616 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae954ac-e066-4bf5-9b09-7864dd1e6ab5 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:49.996326 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 531/1053] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:52.205588 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0bc1-63ce-49dd-9104-1ebc78f7ed0f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:52.206211 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0bc1-63ce-49dd-9104-1ebc78f7ed0f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:52.206211 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0bc1-63ce-49dd-9104-1ebc78f7ed0f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:52.207828 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 523/1054] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:52.214203 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dead5-045a-4aad-8474-685920a8012c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:52.214753 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dead5-045a-4aad-8474-685920a8012c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:52.214753 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dead5-045a-4aad-8474-685920a8012c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:52.281943 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-353dead5-045a-4aad-8474-685920a8012c None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:52.283140 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-353dead5-045a-4aad-8474-685920a8012c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:52.289946 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:52.379972 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 532/1055] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:52.663266 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd0bc1-63ce-49dd-9104-1ebc78f7ed0f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:52.664794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-44fd0bc1-63ce-49dd-9104-1ebc78f7ed0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:52.814653 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd0bc1-63ce-49dd-9104-1ebc78f7ed0f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:52.818736 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:52.933704 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc39b40-3946-48f3-b7d0-06f059839526 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:52.934228 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc39b40-3946-48f3-b7d0-06f059839526 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:52.934507 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc39b40-3946-48f3-b7d0-06f059839526 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:52.934775 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc39b40-3946-48f3-b7d0-06f059839526 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:52.936342 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8fc39b40-3946-48f3-b7d0-06f059839526 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:53.032127 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc39b40-3946-48f3-b7d0-06f059839526 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:53.033748 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc39b40-3946-48f3-b7d0-06f059839526 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:53.052805 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 524/1056] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:55.251338 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f344e-af3e-4b94-8bbb-1e9fc0e5020a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:55.251775 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f344e-af3e-4b94-8bbb-1e9fc0e5020a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:55.251949 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f344e-af3e-4b94-8bbb-1e9fc0e5020a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:55.256659 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 533/1057] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:55.259157 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a882d-21b0-4de0-9f2f-d8e85ec4e4a9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:55.259561 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a882d-21b0-4de0-9f2f-d8e85ec4e4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:55.259707 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a882d-21b0-4de0-9f2f-d8e85ec4e4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:55.271477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:55.347643 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1c3a882d-21b0-4de0-9f2f-d8e85ec4e4a9 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:55.350516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1c3a882d-21b0-4de0-9f2f-d8e85ec4e4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:55.486222 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 525/1058] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:55.811917 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f344e-af3e-4b94-8bbb-1e9fc0e5020a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:55.813971 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-868f344e-af3e-4b94-8bbb-1e9fc0e5020a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:55.968929 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:55.980803 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f344e-af3e-4b94-8bbb-1e9fc0e5020a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:56.107514 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53271e-4d72-4d83-8c2a-86fcb66f6216 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:56.108041 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53271e-4d72-4d83-8c2a-86fcb66f6216 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:56.108349 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53271e-4d72-4d83-8c2a-86fcb66f6216 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:56.108629 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53271e-4d72-4d83-8c2a-86fcb66f6216 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:56.110613 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ca53271e-4d72-4d83-8c2a-86fcb66f6216 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:56.210692 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53271e-4d72-4d83-8c2a-86fcb66f6216 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:56.213457 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53271e-4d72-4d83-8c2a-86fcb66f6216 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:56.223788 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:56.233698 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 534/1059] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:58.758510 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659835-f925-4709-a89b-2b313045f236 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:58.759146 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659835-f925-4709-a89b-2b313045f236 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:58.759146 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659835-f925-4709-a89b-2b313045f236 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:58.760568 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 526/1060] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:58.766341 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72aa2e4-cd6e-4cd4-a2a8-d2e7dbe677fc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:58.767016 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72aa2e4-cd6e-4cd4-a2a8-d2e7dbe677fc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:58.767268 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72aa2e4-cd6e-4cd4-a2a8-d2e7dbe677fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:58.845292 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f72aa2e4-cd6e-4cd4-a2a8-d2e7dbe677fc None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:58.846526 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f72aa2e4-cd6e-4cd4-a2a8-d2e7dbe677fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:58.967913 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 535/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:19:59.260110 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659835-f925-4709-a89b-2b313045f236 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:59.263228 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-86659835-f925-4709-a89b-2b313045f236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:59.362391 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:59.429303 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659835-f925-4709-a89b-2b313045f236 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:59.548789 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb5782-c0d7-45a0-8251-d0fe891de1a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:59.549323 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb5782-c0d7-45a0-8251-d0fe891de1a5 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:59.549601 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb5782-c0d7-45a0-8251-d0fe891de1a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:59.549872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb5782-c0d7-45a0-8251-d0fe891de1a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:59.551517 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-65eb5782-c0d7-45a0-8251-d0fe891de1a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:59.604366 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:59.647332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb5782-c0d7-45a0-8251-d0fe891de1a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:59.649313 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb5782-c0d7-45a0-8251-d0fe891de1a5 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:59.667039 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 527/1062] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:01.828893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7efe3-de3b-45b2-9a5e-1e2f613f2ab3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:01.829471 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7efe3-de3b-45b2-9a5e-1e2f613f2ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:01.829471 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7efe3-de3b-45b2-9a5e-1e2f613f2ab3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:01.830768 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 536/1063] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:01.836477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981f154-e1a5-41f6-ae9f-74ccd198333a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:01.836937 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981f154-e1a5-41f6-ae9f-74ccd198333a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:01.837075 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981f154-e1a5-41f6-ae9f-74ccd198333a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:01.904929 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a981f154-e1a5-41f6-ae9f-74ccd198333a None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:01.906043 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a981f154-e1a5-41f6-ae9f-74ccd198333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:02.019258 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 528/1064] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:02.306818 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7efe3-de3b-45b2-9a5e-1e2f613f2ab3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:02.308340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6ca7efe3-de3b-45b2-9a5e-1e2f613f2ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:02.386520 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:02.457798 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7efe3-de3b-45b2-9a5e-1e2f613f2ab3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:02.571094 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf592656-f335-4b1f-aca2-56ff4103a51d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:02.571651 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf592656-f335-4b1f-aca2-56ff4103a51d service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:02.571928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf592656-f335-4b1f-aca2-56ff4103a51d service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:02.572197 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf592656-f335-4b1f-aca2-56ff4103a51d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:02.573797 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bf592656-f335-4b1f-aca2-56ff4103a51d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:02.612434 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:02.671485 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf592656-f335-4b1f-aca2-56ff4103a51d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:02.673169 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf592656-f335-4b1f-aca2-56ff4103a51d service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:02.691388 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 537/1065] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:03.303060 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab36192-ca67-43ab-b018-29afa0799baf None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:03.303629 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab36192-ca67-43ab-b018-29afa0799baf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:03.303629 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab36192-ca67-43ab-b018-29afa0799baf None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:03.304957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 529/1066] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:03.310276 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b02dd-a304-47c4-bae0-29c6061f200b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:03.310681 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b02dd-a304-47c4-bae0-29c6061f200b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:03.310986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b02dd-a304-47c4-bae0-29c6061f200b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:03.378841 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-537b02dd-a304-47c4-bae0-29c6061f200b None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:03.380056 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-537b02dd-a304-47c4-bae0-29c6061f200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:03.479391 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 538/1067] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:03.780166 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6a7a3-5268-4784-a628-3d30f990a761 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:03.780523 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6a7a3-5268-4784-a628-3d30f990a761 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:03.780957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6a7a3-5268-4784-a628-3d30f990a761 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:03.782291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 530/1068] 10.222.0.15 () {54 vars in 962 bytes} [Wed Nov 2 07:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:03.787871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b26df0-0dff-4f4e-b052-f633db273dc8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:03.788157 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b26df0-0dff-4f4e-b052-f633db273dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:03.788435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b26df0-0dff-4f4e-b052-f633db273dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:03.804486 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:03.868460 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c7b26df0-0dff-4f4e-b052-f633db273dc8 None None] MFA Rules not processed for user `7a5b837bc9c04fc5a53077d152f4d6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:03.869699 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c7b26df0-0dff-4f4e-b052-f633db273dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:03.946383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 539/1069] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Nov 2 07:20:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:20:03.954137 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6a7a3-5268-4784-a628-3d30f990a761 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:03.955780 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-11c6a7a3-5268-4784-a628-3d30f990a761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:03.987810 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:04.061402 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6a7a3-5268-4784-a628-3d30f990a761 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:04.135243 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40a614-566f-4b02-9fee-626df3c68ffd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a5b837bc9c04fc5a53077d152f4d6d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:04.135850 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40a614-566f-4b02-9fee-626df3c68ffd service glance] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:04.136126 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40a614-566f-4b02-9fee-626df3c68ffd service glance] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:04.136391 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40a614-566f-4b02-9fee-626df3c68ffd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:04.138007 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-da40a614-566f-4b02-9fee-626df3c68ffd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:04.215964 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:04.248570 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40a614-566f-4b02-9fee-626df3c68ffd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:04.249994 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40a614-566f-4b02-9fee-626df3c68ffd service glance] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:04.268646 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 531/1070] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:05.010012 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc333602-f6dc-4c19-b5ba-6347bc339a8d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:05.010577 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc333602-f6dc-4c19-b5ba-6347bc339a8d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:05.010795 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc333602-f6dc-4c19-b5ba-6347bc339a8d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:05.012849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 540/1071] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:05.020360 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6375f4-e5c8-46f1-bc7b-48887ae78949 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:05.021922 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6375f4-e5c8-46f1-bc7b-48887ae78949 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:05.022345 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6375f4-e5c8-46f1-bc7b-48887ae78949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:05.041010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333e376-0591-4c5a-9ff5-d155596e6a4b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:05.041371 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333e376-0591-4c5a-9ff5-d155596e6a4b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:05.041779 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333e376-0591-4c5a-9ff5-d155596e6a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:05.101516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-df6375f4-e5c8-46f1-bc7b-48887ae78949 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:05.102656 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-df6375f4-e5c8-46f1-bc7b-48887ae78949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:05.114282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6333e376-0591-4c5a-9ff5-d155596e6a4b None None] MFA Rules not processed for user `b67332ed32044f94a1b3a6a6e6bbb348`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:05.115506 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6333e376-0591-4c5a-9ff5-d155596e6a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:05.171899 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:05.200343 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 532/1072] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:05.203927 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 541/1073] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Nov 2 07:20:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:20:05.253251 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f2fd8-f299-4614-b4c0-e952437f84ef None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:05.253564 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f2fd8-f299-4614-b4c0-e952437f84ef None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:05.253841 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f2fd8-f299-4614-b4c0-e952437f84ef None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:05.255213 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 533/1074] 10.222.0.15 () {54 vars in 943 bytes} [Wed Nov 2 07:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:05.262188 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bed0cc-cd4b-44f4-859f-b9de4ca4ddac None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:05.262567 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bed0cc-cd4b-44f4-859f-b9de4ca4ddac None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:05.262912 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bed0cc-cd4b-44f4-859f-b9de4ca4ddac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:05.339712 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-03bed0cc-cd4b-44f4-859f-b9de4ca4ddac None None] MFA Rules not processed for user `7acc23ec5ccf4756a9e00403116ffe6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:05.340965 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-03bed0cc-cd4b-44f4-859f-b9de4ca4ddac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:05.449135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 542/1075] 10.222.0.15 () {58 vars in 1040 bytes} [Wed Nov 2 07:20:05 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:20:05.458435 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f2fd8-f299-4614-b4c0-e952437f84ef None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:05.460351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5d8f2fd8-f299-4614-b4c0-e952437f84ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:05.494680 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bed0cc-cd4b-44f4-859f-b9de4ca4ddac None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:05.495901 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-03bed0cc-cd4b-44f4-859f-b9de4ca4ddac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:05.502121 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:05.527819 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:05.597432 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bed0cc-cd4b-44f4-859f-b9de4ca4ddac None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:05.601186 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f2fd8-f299-4614-b4c0-e952437f84ef None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:05.712451 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b219924-4d0e-4878-b7b9-c3123df98c6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7acc23ec5ccf4756a9e00403116ffe6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:05.713018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b219924-4d0e-4878-b7b9-c3123df98c6c service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:05.713415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b219924-4d0e-4878-b7b9-c3123df98c6c service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:05.713692 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b219924-4d0e-4878-b7b9-c3123df98c6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:05.714290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fda9fb-dcf6-46ae-b4c5-35070de659ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:05.714898 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fda9fb-dcf6-46ae-b4c5-35070de659ee service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:05.715168 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fda9fb-dcf6-46ae-b4c5-35070de659ee service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:05.715332 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8b219924-4d0e-4878-b7b9-c3123df98c6c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:05.715430 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fda9fb-dcf6-46ae-b4c5-35070de659ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:05.717015 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-32fda9fb-dcf6-46ae-b4c5-35070de659ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:05.737711 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:05.760682 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:05.802856 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b219924-4d0e-4878-b7b9-c3123df98c6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:05.804463 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b219924-4d0e-4878-b7b9-c3123df98c6c service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:05.808826 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fda9fb-dcf6-46ae-b4c5-35070de659ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:05.810402 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fda9fb-dcf6-46ae-b4c5-35070de659ee service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:05.812791 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 534/1076] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Nov 2 07:20:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:20:05.830883 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 543/1077] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:20:08.077242 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ffb29-3542-4e66-a6c4-1b541d329f86 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:08.077840 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ffb29-3542-4e66-a6c4-1b541d329f86 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:08.077840 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ffb29-3542-4e66-a6c4-1b541d329f86 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:08.079146 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 535/1078] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:08.085473 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d255fe-b369-45e7-83b4-83a4894f267d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:08.085939 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d255fe-b369-45e7-83b4-83a4894f267d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:08.086044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d255fe-b369-45e7-83b4-83a4894f267d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:08.161529 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-57d255fe-b369-45e7-83b4-83a4894f267d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:08.163650 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-57d255fe-b369-45e7-83b4-83a4894f267d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:08.277827 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 544/1079] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:08.565780 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ffb29-3542-4e66-a6c4-1b541d329f86 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:08.567334 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4e0ffb29-3542-4e66-a6c4-1b541d329f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:08.727858 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ffb29-3542-4e66-a6c4-1b541d329f86 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:08.745504 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:08.850153 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6b550-503b-432c-8942-9546290766c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:08.850684 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6b550-503b-432c-8942-9546290766c6 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:08.850968 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6b550-503b-432c-8942-9546290766c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:08.851241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6b550-503b-432c-8942-9546290766c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:08.852958 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9cd6b550-503b-432c-8942-9546290766c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:08.951504 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6b550-503b-432c-8942-9546290766c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:08.953167 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6b550-503b-432c-8942-9546290766c6 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:08.976654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 536/1080] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:11.168174 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ba6f8-ec95-4c75-aa69-2999b95d0fe0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:11.168782 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ba6f8-ec95-4c75-aa69-2999b95d0fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:11.168889 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ba6f8-ec95-4c75-aa69-2999b95d0fe0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:11.170444 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 545/1081] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:11.175529 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385486b4-db6a-47d0-bcf0-8d96b1189f69 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:11.175976 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385486b4-db6a-47d0-bcf0-8d96b1189f69 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:11.176087 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385486b4-db6a-47d0-bcf0-8d96b1189f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:11.205372 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:11.249720 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-385486b4-db6a-47d0-bcf0-8d96b1189f69 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:11.251470 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-385486b4-db6a-47d0-bcf0-8d96b1189f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:11.372416 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 537/1082] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:11.663096 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ba6f8-ec95-4c75-aa69-2999b95d0fe0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:11.664650 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-361ba6f8-ec95-4c75-aa69-2999b95d0fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:11.686094 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:11.838097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ba6f8-ec95-4c75-aa69-2999b95d0fe0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:11.951027 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:11.978713 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337aa10d-1eac-4e38-a9c0-7dcfc662afea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:11.979333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aa10d-1eac-4e38-a9c0-7dcfc662afea service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:11.979615 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aa10d-1eac-4e38-a9c0-7dcfc662afea service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:11.979889 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aa10d-1eac-4e38-a9c0-7dcfc662afea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:11.981542 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-337aa10d-1eac-4e38-a9c0-7dcfc662afea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:12.092807 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337aa10d-1eac-4e38-a9c0-7dcfc662afea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:12.094405 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337aa10d-1eac-4e38-a9c0-7dcfc662afea service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:12.115453 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 546/1083] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:14.369469 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48aca8-0bc4-4dcf-983b-625766c04ea5 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:14.370003 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48aca8-0bc4-4dcf-983b-625766c04ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:14.370003 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48aca8-0bc4-4dcf-983b-625766c04ea5 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:14.371288 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 538/1084] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:14.376615 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8f249-4f94-467d-bf2d-f06918e8b996 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:14.377044 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8f249-4f94-467d-bf2d-f06918e8b996 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:14.377220 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8f249-4f94-467d-bf2d-f06918e8b996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:14.449274 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-88b8f249-4f94-467d-bf2d-f06918e8b996 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:14.452127 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-88b8f249-4f94-467d-bf2d-f06918e8b996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:14.508469 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:14.558561 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 547/1085] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:20:14.856326 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48aca8-0bc4-4dcf-983b-625766c04ea5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:14.858096 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3a48aca8-0bc4-4dcf-983b-625766c04ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:14.957737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:15.019848 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48aca8-0bc4-4dcf-983b-625766c04ea5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:15.149366 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4b217-d0a4-4e83-b9b8-958064dfb74e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:15.150073 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b217-d0a4-4e83-b9b8-958064dfb74e service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:15.150448 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b217-d0a4-4e83-b9b8-958064dfb74e service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:15.150779 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b217-d0a4-4e83-b9b8-958064dfb74e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:15.152675 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-39f4b217-d0a4-4e83-b9b8-958064dfb74e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:15.220972 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:15.256302 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4b217-d0a4-4e83-b9b8-958064dfb74e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:15.257957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4b217-d0a4-4e83-b9b8-958064dfb74e service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:15.276801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 539/1086] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:24.557295 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=19 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:25.274477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=18 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:28.689928 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d744-1dc6-4a39-9e60-6704ee72e1de None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:28.690485 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d744-1dc6-4a39-9e60-6704ee72e1de None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:28.690485 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d744-1dc6-4a39-9e60-6704ee72e1de None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:28.691842 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 548/1087] 10.222.0.15 () {54 vars in 918 bytes} [Wed Nov 2 07:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:28.697490 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0799852-ce8c-4e37-90b1-7380f9adfcc7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:28.697925 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0799852-ce8c-4e37-90b1-7380f9adfcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:28.698035 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0799852-ce8c-4e37-90b1-7380f9adfcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:28.772702 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d0799852-ce8c-4e37-90b1-7380f9adfcc7 None None] MFA Rules not processed for user `19c9d282c1a74c25939c03e8dea6a345`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:28.773909 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d0799852-ce8c-4e37-90b1-7380f9adfcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:28.878027 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 540/1088] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Nov 2 07:20:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:28.892693 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942687-7d2e-4d90-8b99-161b42fa1f25 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:28.892930 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942687-7d2e-4d90-8b99-161b42fa1f25 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:28.893207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942687-7d2e-4d90-8b99-161b42fa1f25 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:28.894531 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 549/1089] 10.222.0.15 () {54 vars in 918 bytes} [Wed Nov 2 07:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:28.899488 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7dc8c-2bee-44fd-bfae-a08ac3efbd0e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:28.899771 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7dc8c-2bee-44fd-bfae-a08ac3efbd0e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:28.900041 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7dc8c-2bee-44fd-bfae-a08ac3efbd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:28.972226 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:28.975566 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7cc7dc8c-2bee-44fd-bfae-a08ac3efbd0e None None] MFA Rules not processed for user `00ab674b018e474ab363e8e17d225e27`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:28.977298 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7cc7dc8c-2bee-44fd-bfae-a08ac3efbd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:29.091616 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 541/1090] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Nov 2 07:20:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:31.656459 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166533b-1317-43b3-aba9-c242791f3bbd None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:31.657339 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166533b-1317-43b3-aba9-c242791f3bbd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:31.657619 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166533b-1317-43b3-aba9-c242791f3bbd None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:31.658984 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 550/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:31.664718 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b1dc7-6cbb-4ee9-8739-196da2ac78da None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:31.665134 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b1dc7-6cbb-4ee9-8739-196da2ac78da None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:31.665269 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b1dc7-6cbb-4ee9-8739-196da2ac78da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:31.730813 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c09b1dc7-6cbb-4ee9-8739-196da2ac78da None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:31.732013 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c09b1dc7-6cbb-4ee9-8739-196da2ac78da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:31.785294 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:31.827111 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 542/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:32.112641 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166533b-1317-43b3-aba9-c242791f3bbd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:32.114678 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d166533b-1317-43b3-aba9-c242791f3bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:32.258934 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166533b-1317-43b3-aba9-c242791f3bbd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:32.329607 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:32.370333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578c6b9-eec1-4e72-8c94-326e95bdadc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:32.370951 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578c6b9-eec1-4e72-8c94-326e95bdadc3 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:32.371233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578c6b9-eec1-4e72-8c94-326e95bdadc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:32.371506 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578c6b9-eec1-4e72-8c94-326e95bdadc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:32.373147 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1578c6b9-eec1-4e72-8c94-326e95bdadc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:32.460826 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578c6b9-eec1-4e72-8c94-326e95bdadc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:32.462410 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578c6b9-eec1-4e72-8c94-326e95bdadc3 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:32.480591 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 551/1093] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:33.599130 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecc23a-7735-4caa-ae76-eec1b871f977 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:33.599690 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecc23a-7735-4caa-ae76-eec1b871f977 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:33.599795 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecc23a-7735-4caa-ae76-eec1b871f977 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:33.601101 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 543/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:33.605844 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f630bca-4ea6-447a-a89e-45b61b9fba3b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:33.606264 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f630bca-4ea6-447a-a89e-45b61b9fba3b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:33.606401 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f630bca-4ea6-447a-a89e-45b61b9fba3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:33.683633 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0f630bca-4ea6-447a-a89e-45b61b9fba3b None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:33.684856 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0f630bca-4ea6-447a-a89e-45b61b9fba3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:33.711969 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:33.780207 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 552/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:33.940868 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecc23a-7735-4caa-ae76-eec1b871f977 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:33.942893 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-85ecc23a-7735-4caa-ae76-eec1b871f977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:34.085424 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecc23a-7735-4caa-ae76-eec1b871f977 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:34.123959 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:34.196178 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2438203-d12b-414f-89e5-425f8443f234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:34.196747 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2438203-d12b-414f-89e5-425f8443f234 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:34.197048 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2438203-d12b-414f-89e5-425f8443f234 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:34.197316 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2438203-d12b-414f-89e5-425f8443f234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:34.198888 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b2438203-d12b-414f-89e5-425f8443f234 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:34.286989 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2438203-d12b-414f-89e5-425f8443f234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:34.288587 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2438203-d12b-414f-89e5-425f8443f234 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:34.306367 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 544/1096] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:35.417152 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59555-4048-4d89-b69e-d5d1dc51c9ac None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:35.417610 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59555-4048-4d89-b69e-d5d1dc51c9ac None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:35.417715 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59555-4048-4d89-b69e-d5d1dc51c9ac None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:35.419051 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 553/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:35.423865 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9177163-5937-45fa-b3dd-f136890c7d63 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:35.424291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9177163-5937-45fa-b3dd-f136890c7d63 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:35.424482 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9177163-5937-45fa-b3dd-f136890c7d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:35.479436 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:35.501384 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a9177163-5937-45fa-b3dd-f136890c7d63 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:35.502590 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a9177163-5937-45fa-b3dd-f136890c7d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:35.597983 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 545/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:35.748358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d59555-4048-4d89-b69e-d5d1dc51c9ac None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:35.749841 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-26d59555-4048-4d89-b69e-d5d1dc51c9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:35.896610 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d59555-4048-4d89-b69e-d5d1dc51c9ac None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:35.908465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:36.009298 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdef0d6-5569-4a0d-95c1-4c8e7d5f91ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:36.009871 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdef0d6-5569-4a0d-95c1-4c8e7d5f91ba service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:36.010153 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdef0d6-5569-4a0d-95c1-4c8e7d5f91ba service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:36.010428 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdef0d6-5569-4a0d-95c1-4c8e7d5f91ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:36.012028 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7fdef0d6-5569-4a0d-95c1-4c8e7d5f91ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:36.100313 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdef0d6-5569-4a0d-95c1-4c8e7d5f91ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:36.101932 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdef0d6-5569-4a0d-95c1-4c8e7d5f91ba service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:36.119546 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 554/1099] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:37.216200 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd83a59-7c7b-419b-8e0b-a5bc5e4fb658 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:37.216771 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd83a59-7c7b-419b-8e0b-a5bc5e4fb658 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:37.216771 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd83a59-7c7b-419b-8e0b-a5bc5e4fb658 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:37.218148 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 546/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:37.223525 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e038142-f382-45c3-b83a-fd6a35f27c09 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:37.223901 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e038142-f382-45c3-b83a-fd6a35f27c09 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:37.224076 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e038142-f382-45c3-b83a-fd6a35f27c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:37.243511 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:37.299026 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5e038142-f382-45c3-b83a-fd6a35f27c09 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:37.300222 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5e038142-f382-45c3-b83a-fd6a35f27c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:37.396290 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 555/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:37.546447 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd83a59-7c7b-419b-8e0b-a5bc5e4fb658 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:37.547884 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0cd83a59-7c7b-419b-8e0b-a5bc5e4fb658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:37.657231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:37.692828 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd83a59-7c7b-419b-8e0b-a5bc5e4fb658 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:37.806267 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541de55-5499-4337-8d1f-20ca54036aaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:37.806808 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541de55-5499-4337-8d1f-20ca54036aaa service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:37.807088 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541de55-5499-4337-8d1f-20ca54036aaa service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:37.807358 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541de55-5499-4337-8d1f-20ca54036aaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:37.808984 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5541de55-5499-4337-8d1f-20ca54036aaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:37.886448 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:37.900645 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541de55-5499-4337-8d1f-20ca54036aaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:37.902302 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541de55-5499-4337-8d1f-20ca54036aaa service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:37.919893 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 547/1102] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:39.005109 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcd168-5a06-44d0-8d73-c3bd1f1ad474 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:39.005677 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcd168-5a06-44d0-8d73-c3bd1f1ad474 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:39.005677 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcd168-5a06-44d0-8d73-c3bd1f1ad474 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:39.007005 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 556/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:20:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:39.011934 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360a413-46be-469c-8927-132fffc61d82 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:39.012326 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360a413-46be-469c-8927-132fffc61d82 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:39.012476 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360a413-46be-469c-8927-132fffc61d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:39.086530 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2360a413-46be-469c-8927-132fffc61d82 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:39.087723 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2360a413-46be-469c-8927-132fffc61d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:39.183878 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 548/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:39.332985 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcd168-5a06-44d0-8d73-c3bd1f1ad474 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:39.334705 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-89bcd168-5a06-44d0-8d73-c3bd1f1ad474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:39.418383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:39.481248 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcd168-5a06-44d0-8d73-c3bd1f1ad474 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:39.594451 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d25a32-b70e-4c55-9d8f-1870024aab54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:39.594981 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d25a32-b70e-4c55-9d8f-1870024aab54 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:39.595261 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d25a32-b70e-4c55-9d8f-1870024aab54 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:39.595533 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d25a32-b70e-4c55-9d8f-1870024aab54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:39.597145 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-81d25a32-b70e-4c55-9d8f-1870024aab54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:39.646704 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:39.688270 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d25a32-b70e-4c55-9d8f-1870024aab54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:39.689882 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d25a32-b70e-4c55-9d8f-1870024aab54 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:39.707986 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 557/1105] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:40.733949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deca261-b17e-4923-a5fd-0971ef26d7fb None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:40.734501 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deca261-b17e-4923-a5fd-0971ef26d7fb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:40.734501 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deca261-b17e-4923-a5fd-0971ef26d7fb None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:40.735837 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 549/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:20:40.740726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97806dd-f360-4e64-b5e7-65c4d3f5b26c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:40.741146 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97806dd-f360-4e64-b5e7-65c4d3f5b26c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:40.741272 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97806dd-f360-4e64-b5e7-65c4d3f5b26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:40.820141 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b97806dd-f360-4e64-b5e7-65c4d3f5b26c None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:40.821388 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b97806dd-f360-4e64-b5e7-65c4d3f5b26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:40.918022 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 558/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:41.066295 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deca261-b17e-4923-a5fd-0971ef26d7fb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:41.068388 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8deca261-b17e-4923-a5fd-0971ef26d7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:41.117603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:41.212291 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deca261-b17e-4923-a5fd-0971ef26d7fb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:41.322195 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75713e29-c2af-458e-a486-29eb5b613124 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:41.322790 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75713e29-c2af-458e-a486-29eb5b613124 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:41.323066 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75713e29-c2af-458e-a486-29eb5b613124 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:41.323334 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75713e29-c2af-458e-a486-29eb5b613124 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:41.324935 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-75713e29-c2af-458e-a486-29eb5b613124 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:41.331814 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:41.415165 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75713e29-c2af-458e-a486-29eb5b613124 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:41.416827 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75713e29-c2af-458e-a486-29eb5b613124 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:41.434980 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 550/1108] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:42.470941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f212073-40e0-484f-84c0-54fc72011850 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:42.471546 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f212073-40e0-484f-84c0-54fc72011850 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:42.471546 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f212073-40e0-484f-84c0-54fc72011850 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:42.472895 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 559/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:42.477848 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd8d4a-de0e-43da-b343-38321947c8e5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:42.478299 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd8d4a-de0e-43da-b343-38321947c8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:42.478396 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd8d4a-de0e-43da-b343-38321947c8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:42.552491 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ecbd8d4a-de0e-43da-b343-38321947c8e5 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:42.553740 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ecbd8d4a-de0e-43da-b343-38321947c8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:42.620515 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:42.649711 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 551/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:42.797010 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f212073-40e0-484f-84c0-54fc72011850 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:42.798489 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4f212073-40e0-484f-84c0-54fc72011850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:42.815779 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:42.945818 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f212073-40e0-484f-84c0-54fc72011850 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:43.034683 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:43.059820 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cb47a-603e-4f11-9d10-414c4dbffb23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:43.060365 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cb47a-603e-4f11-9d10-414c4dbffb23 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:43.060680 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cb47a-603e-4f11-9d10-414c4dbffb23 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:43.060956 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cb47a-603e-4f11-9d10-414c4dbffb23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:43.062553 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e72cb47a-603e-4f11-9d10-414c4dbffb23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:43.151346 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cb47a-603e-4f11-9d10-414c4dbffb23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:43.152949 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cb47a-603e-4f11-9d10-414c4dbffb23 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:43.170422 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 560/1111] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:44.193525 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59335000-7aa0-459c-bb4c-82117abb210d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:44.194046 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59335000-7aa0-459c-bb4c-82117abb210d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:44.194046 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59335000-7aa0-459c-bb4c-82117abb210d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:44.195311 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 552/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:44.200273 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9a513-4bef-437e-b46d-2b970e9562e9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:44.200685 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9a513-4bef-437e-b46d-2b970e9562e9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:44.200845 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9a513-4bef-437e-b46d-2b970e9562e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:44.277319 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7ea9a513-4bef-437e-b46d-2b970e9562e9 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:44.278518 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7ea9a513-4bef-437e-b46d-2b970e9562e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:44.319542 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:44.374726 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 561/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:44.524973 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59335000-7aa0-459c-bb4c-82117abb210d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:44.526408 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-59335000-7aa0-459c-bb4c-82117abb210d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:44.671007 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59335000-7aa0-459c-bb4c-82117abb210d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:44.724801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:44.782458 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e1394-6d37-46d1-8046-2b8cebd81917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:44.782998 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e1394-6d37-46d1-8046-2b8cebd81917 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:44.783277 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e1394-6d37-46d1-8046-2b8cebd81917 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:44.783547 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e1394-6d37-46d1-8046-2b8cebd81917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:44.785715 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b63e1394-6d37-46d1-8046-2b8cebd81917 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:44.873481 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e1394-6d37-46d1-8046-2b8cebd81917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:44.875056 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e1394-6d37-46d1-8046-2b8cebd81917 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:44.892673 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 553/1114] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:45.924650 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3d371-5a58-4be7-8153-27a9dcbb7ee9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:45.925183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3d371-5a58-4be7-8153-27a9dcbb7ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:45.925183 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3d371-5a58-4be7-8153-27a9dcbb7ee9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:45.926453 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 562/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:45.931255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6f042-d312-4b1c-9080-dc4decda18cb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:45.931674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6f042-d312-4b1c-9080-dc4decda18cb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:45.931787 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6f042-d312-4b1c-9080-dc4decda18cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:46.007983 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-59a6f042-d312-4b1c-9080-dc4decda18cb None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:46.009234 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-59a6f042-d312-4b1c-9080-dc4decda18cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:46.011224 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:46.104556 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 554/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:46.252876 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3d371-5a58-4be7-8153-27a9dcbb7ee9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:46.254892 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a2c3d371-5a58-4be7-8153-27a9dcbb7ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:46.398808 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3d371-5a58-4be7-8153-27a9dcbb7ee9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:46.425058 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:46.510932 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42f929-f198-4157-ab63-2d687100a660 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:46.511464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42f929-f198-4157-ab63-2d687100a660 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:46.511739 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42f929-f198-4157-ab63-2d687100a660 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:46.512008 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42f929-f198-4157-ab63-2d687100a660 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:46.513642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ef42f929-f198-4157-ab63-2d687100a660 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:46.600854 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42f929-f198-4157-ab63-2d687100a660 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:46.602419 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42f929-f198-4157-ab63-2d687100a660 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:46.620286 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 563/1117] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:48.696460 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8391977-f7d0-4143-ba02-8886ac29d6d2 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:48.697018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8391977-f7d0-4143-ba02-8886ac29d6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:48.697126 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8391977-f7d0-4143-ba02-8886ac29d6d2 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:48.698386 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 555/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:48.703932 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662c334-e202-49dc-b6e8-27fcef49b3c3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:48.704375 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662c334-e202-49dc-b6e8-27fcef49b3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:48.704481 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662c334-e202-49dc-b6e8-27fcef49b3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:48.745965 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:48.773163 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0662c334-e202-49dc-b6e8-27fcef49b3c3 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:48.774369 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0662c334-e202-49dc-b6e8-27fcef49b3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:48.869618 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 564/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:49.156911 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8391977-f7d0-4143-ba02-8886ac29d6d2 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:49.158429 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d8391977-f7d0-4143-ba02-8886ac29d6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:49.259925 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8391977-f7d0-4143-ba02-8886ac29d6d2 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:49.292018 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:49.334577 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64188ebd-e97a-4488-9d71-89a906ac7fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:49.335176 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188ebd-e97a-4488-9d71-89a906ac7fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:49.335452 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188ebd-e97a-4488-9d71-89a906ac7fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:49.335720 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188ebd-e97a-4488-9d71-89a906ac7fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:49.337496 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-64188ebd-e97a-4488-9d71-89a906ac7fa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:49.447614 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64188ebd-e97a-4488-9d71-89a906ac7fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:49.449094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64188ebd-e97a-4488-9d71-89a906ac7fa7 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:49.467326 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 556/1120] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:51.745437 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac6e3d-8a40-40e5-bc66-782118bd08ee None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:51.745979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac6e3d-8a40-40e5-bc66-782118bd08ee None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:51.745979 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac6e3d-8a40-40e5-bc66-782118bd08ee None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:51.747351 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 565/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:51.752825 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b78ad6-c61c-4eb5-a80c-4b2240cae2e5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:51.753250 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b78ad6-c61c-4eb5-a80c-4b2240cae2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:51.753376 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b78ad6-c61c-4eb5-a80c-4b2240cae2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:51.819105 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-91b78ad6-c61c-4eb5-a80c-4b2240cae2e5 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:51.820310 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-91b78ad6-c61c-4eb5-a80c-4b2240cae2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:51.833560 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:51.917070 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 557/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:52.200791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac6e3d-8a40-40e5-bc66-782118bd08ee None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:52.202343 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-45ac6e3d-8a40-40e5-bc66-782118bd08ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:52.304118 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac6e3d-8a40-40e5-bc66-782118bd08ee None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:52.317084 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:52.377742 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5029a-384b-43f6-ae23-156d29bee2f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:52.378340 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5029a-384b-43f6-ae23-156d29bee2f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:52.378615 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5029a-384b-43f6-ae23-156d29bee2f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:52.378882 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5029a-384b-43f6-ae23-156d29bee2f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:52.381020 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-74e5029a-384b-43f6-ae23-156d29bee2f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:52.487855 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5029a-384b-43f6-ae23-156d29bee2f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:52.489784 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5029a-384b-43f6-ae23-156d29bee2f9 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:52.507125 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 566/1123] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:54.943229 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08af58-dd99-4d48-aa0c-1c69540e54fa None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:54.943737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08af58-dd99-4d48-aa0c-1c69540e54fa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:54.943737 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08af58-dd99-4d48-aa0c-1c69540e54fa None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:54.945053 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 558/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:54.950405 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9bf48-329d-487b-adb0-15c42484adc3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:54.950834 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9bf48-329d-487b-adb0-15c42484adc3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:54.950971 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9bf48-329d-487b-adb0-15c42484adc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:54.999625 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:55.020766 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-48d9bf48-329d-487b-adb0-15c42484adc3 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:55.022004 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-48d9bf48-329d-487b-adb0-15c42484adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:55.117966 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 567/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:55.400101 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08af58-dd99-4d48-aa0c-1c69540e54fa None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:55.401665 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5e08af58-dd99-4d48-aa0c-1c69540e54fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:55.505356 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08af58-dd99-4d48-aa0c-1c69540e54fa None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:55.544712 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:55.578257 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae9b81-87d8-45d4-a3cd-ad54741aca9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:55.578859 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae9b81-87d8-45d4-a3cd-ad54741aca9f service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:55.579133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae9b81-87d8-45d4-a3cd-ad54741aca9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:55.579400 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae9b81-87d8-45d4-a3cd-ad54741aca9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:55.581011 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-38ae9b81-87d8-45d4-a3cd-ad54741aca9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:55.690412 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae9b81-87d8-45d4-a3cd-ad54741aca9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:55.691760 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae9b81-87d8-45d4-a3cd-ad54741aca9f service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:55.709418 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 559/1126] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:57.956095 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d19533-0b27-424d-805d-e1b938ef16e6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:57.956495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d19533-0b27-424d-805d-e1b938ef16e6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:57.956684 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d19533-0b27-424d-805d-e1b938ef16e6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:57.958904 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 568/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:57.964102 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903aa5cf-5703-4e04-8eac-5457a536da6c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:57.964512 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903aa5cf-5703-4e04-8eac-5457a536da6c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:57.964683 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903aa5cf-5703-4e04-8eac-5457a536da6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:58.029823 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-903aa5cf-5703-4e04-8eac-5457a536da6c None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:58.031007 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-903aa5cf-5703-4e04-8eac-5457a536da6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:58.050892 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:58.125533 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 560/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:58.409298 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d19533-0b27-424d-805d-e1b938ef16e6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:58.411415 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e5d19533-0b27-424d-805d-e1b938ef16e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:58.514796 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d19533-0b27-424d-805d-e1b938ef16e6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:58.534893 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:58.588153 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b0b46-239c-429a-aa3e-ca51c544bb94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:58.588784 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b0b46-239c-429a-aa3e-ca51c544bb94 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:58.589063 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b0b46-239c-429a-aa3e-ca51c544bb94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:58.589331 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b0b46-239c-429a-aa3e-ca51c544bb94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:58.590910 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8b2b0b46-239c-429a-aa3e-ca51c544bb94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:58.698140 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b0b46-239c-429a-aa3e-ca51c544bb94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:58.699497 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b0b46-239c-429a-aa3e-ca51c544bb94 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:58.717809 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 569/1129] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:00.927531 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4e76-c342-4238-83c6-bd04097cee93 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:00.928063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4e76-c342-4238-83c6-bd04097cee93 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:00.928063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4e76-c342-4238-83c6-bd04097cee93 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:00.929426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 561/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:00.934780 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c37fb-1ed1-46e9-bb24-eca34c3b2869 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:00.935231 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c37fb-1ed1-46e9-bb24-eca34c3b2869 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:00.935336 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c37fb-1ed1-46e9-bb24-eca34c3b2869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:01.000024 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:01.002772 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-451c37fb-1ed1-46e9-bb24-eca34c3b2869 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:01.004492 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-451c37fb-1ed1-46e9-bb24-eca34c3b2869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:01.099619 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 570/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:01.382924 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd4e76-c342-4238-83c6-bd04097cee93 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:01.384477 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-88dd4e76-c342-4238-83c6-bd04097cee93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:01.487175 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd4e76-c342-4238-83c6-bd04097cee93 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:01.535838 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:01.559801 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbae908-a146-417e-b466-b9ccfe03e3cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:01.560395 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbae908-a146-417e-b466-b9ccfe03e3cb service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:01.560702 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbae908-a146-417e-b466-b9ccfe03e3cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:01.560973 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbae908-a146-417e-b466-b9ccfe03e3cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:01.562538 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2bbae908-a146-417e-b466-b9ccfe03e3cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:01.670861 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbae908-a146-417e-b466-b9ccfe03e3cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:01.672274 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbae908-a146-417e-b466-b9ccfe03e3cb service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:01.690160 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 562/1132] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:03.882084 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732d52d-b50b-4c8d-b7fe-09307d56ef3e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:03.882642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732d52d-b50b-4c8d-b7fe-09307d56ef3e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:03.882642 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732d52d-b50b-4c8d-b7fe-09307d56ef3e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:03.883977 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 571/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:03.889365 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5a2f3-c5f2-4035-8d23-b783e4c1605f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:03.889807 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5a2f3-c5f2-4035-8d23-b783e4c1605f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:03.889916 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5a2f3-c5f2-4035-8d23-b783e4c1605f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:03.954881 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-10f5a2f3-c5f2-4035-8d23-b783e4c1605f None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:03.956039 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-10f5a2f3-c5f2-4035-8d23-b783e4c1605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:03.985640 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:04.051840 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 563/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:04.335195 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732d52d-b50b-4c8d-b7fe-09307d56ef3e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:04.336781 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0732d52d-b50b-4c8d-b7fe-09307d56ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:04.439897 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732d52d-b50b-4c8d-b7fe-09307d56ef3e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:04.468996 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:04.514435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe43fff-bd36-4de7-a596-23c22d4e601d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:04.515017 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43fff-bd36-4de7-a596-23c22d4e601d service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:04.515296 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43fff-bd36-4de7-a596-23c22d4e601d service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:04.515567 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43fff-bd36-4de7-a596-23c22d4e601d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:04.517201 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ffe43fff-bd36-4de7-a596-23c22d4e601d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:04.625168 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe43fff-bd36-4de7-a596-23c22d4e601d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:04.626515 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe43fff-bd36-4de7-a596-23c22d4e601d service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:04.644965 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 572/1135] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:06.836023 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec712fd-794d-4f30-a13b-bc464e44e0ae None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:06.836619 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec712fd-794d-4f30-a13b-bc464e44e0ae None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:06.836619 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec712fd-794d-4f30-a13b-bc464e44e0ae None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:06.837942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 564/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:06.843307 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6015a07-2901-4603-a0bd-2929161de05a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:06.843763 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6015a07-2901-4603-a0bd-2929161de05a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:06.843874 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6015a07-2901-4603-a0bd-2929161de05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:06.909612 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a6015a07-2901-4603-a0bd-2929161de05a None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:06.910771 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a6015a07-2901-4603-a0bd-2929161de05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:06.919055 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:07.007099 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 573/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:07.295894 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec712fd-794d-4f30-a13b-bc464e44e0ae None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:07.297569 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1ec712fd-794d-4f30-a13b-bc464e44e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:07.402388 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec712fd-794d-4f30-a13b-bc464e44e0ae None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:07.458419 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:07.476639 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cf454-15fb-4c94-b7f6-dff1b65512c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:07.477247 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cf454-15fb-4c94-b7f6-dff1b65512c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:07.477525 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cf454-15fb-4c94-b7f6-dff1b65512c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:07.477794 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cf454-15fb-4c94-b7f6-dff1b65512c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:07.479385 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8e1cf454-15fb-4c94-b7f6-dff1b65512c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:07.587552 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cf454-15fb-4c94-b7f6-dff1b65512c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:07.588935 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cf454-15fb-4c94-b7f6-dff1b65512c2 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:07.606537 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 565/1138] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:09.794204 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0bd12-b2dc-4c59-bda4-ad2c36404ea6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:09.794795 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0bd12-b2dc-4c59-bda4-ad2c36404ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:09.794795 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0bd12-b2dc-4c59-bda4-ad2c36404ea6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:09.796084 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 574/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:09.802158 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0aaa2-7166-4daf-9033-3d35b90b109b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:09.802629 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0aaa2-7166-4daf-9033-3d35b90b109b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:09.802752 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0aaa2-7166-4daf-9033-3d35b90b109b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:09.868033 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0af0aaa2-7166-4daf-9033-3d35b90b109b None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:09.869272 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0af0aaa2-7166-4daf-9033-3d35b90b109b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:09.907885 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:09.965631 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 566/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:10.245440 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0bd12-b2dc-4c59-bda4-ad2c36404ea6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:10.247033 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f9c0bd12-b2dc-4c59-bda4-ad2c36404ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:10.353198 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0bd12-b2dc-4c59-bda4-ad2c36404ea6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:10.391438 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:10.427126 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08393b4-69e7-4c3e-887c-1463a2165867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:10.427724 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08393b4-69e7-4c3e-887c-1463a2165867 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:10.427996 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08393b4-69e7-4c3e-887c-1463a2165867 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:10.428262 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08393b4-69e7-4c3e-887c-1463a2165867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:10.429876 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c08393b4-69e7-4c3e-887c-1463a2165867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:10.536963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08393b4-69e7-4c3e-887c-1463a2165867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:10.538282 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08393b4-69e7-4c3e-887c-1463a2165867 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:10.555652 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 575/1141] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:12.706543 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13a18-f44a-46bc-a1e1-a9188ab71f40 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:12.707089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13a18-f44a-46bc-a1e1-a9188ab71f40 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:12.707089 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13a18-f44a-46bc-a1e1-a9188ab71f40 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:12.709306 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 567/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:12.714465 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4c530-d089-4d21-9ef9-974478cca33d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:12.714863 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4c530-d089-4d21-9ef9-974478cca33d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:12.715018 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4c530-d089-4d21-9ef9-974478cca33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:12.781849 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-acd4c530-d089-4d21-9ef9-974478cca33d None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:12.783060 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-acd4c530-d089-4d21-9ef9-974478cca33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:12.800988 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:12.878953 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 576/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:13.161908 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de13a18-f44a-46bc-a1e1-a9188ab71f40 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:13.163423 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8de13a18-f44a-46bc-a1e1-a9188ab71f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:13.265379 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de13a18-f44a-46bc-a1e1-a9188ab71f40 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:13.331220 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:13.339649 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b8037-4d30-4467-a792-fbabb2ff4bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:13.340278 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b8037-4d30-4467-a792-fbabb2ff4bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:13.340579 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b8037-4d30-4467-a792-fbabb2ff4bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:13.340872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b8037-4d30-4467-a792-fbabb2ff4bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:13.342517 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7e1b8037-4d30-4467-a792-fbabb2ff4bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:13.452031 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b8037-4d30-4467-a792-fbabb2ff4bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:13.453407 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b8037-4d30-4467-a792-fbabb2ff4bc8 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:13.471662 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 568/1144] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:15.631566 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ebae3-fded-4027-9570-5070b61e4e0e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:15.632136 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ebae3-fded-4027-9570-5070b61e4e0e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:15.632136 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ebae3-fded-4027-9570-5070b61e4e0e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:15.633519 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 577/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:15.638957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e944-8b9c-4139-b1de-06c0f665b516 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:15.639407 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e944-8b9c-4139-b1de-06c0f665b516 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:15.639510 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e944-8b9c-4139-b1de-06c0f665b516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:15.704389 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1ed6e944-8b9c-4139-b1de-06c0f665b516 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:15.705614 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1ed6e944-8b9c-4139-b1de-06c0f665b516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:15.753446 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:15.802441 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 569/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:21:16.084909 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ebae3-fded-4027-9570-5070b61e4e0e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:16.086435 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-270ebae3-fded-4027-9570-5070b61e4e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:16.187427 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ebae3-fded-4027-9570-5070b61e4e0e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:16.235951 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:16.260836 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effe6e1-0e9c-4e2a-a1e8-a5913e3e3a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:16.261434 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effe6e1-0e9c-4e2a-a1e8-a5913e3e3a38 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:16.261707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effe6e1-0e9c-4e2a-a1e8-a5913e3e3a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:16.261975 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effe6e1-0e9c-4e2a-a1e8-a5913e3e3a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:16.263537 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1effe6e1-0e9c-4e2a-a1e8-a5913e3e3a38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:16.370951 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effe6e1-0e9c-4e2a-a1e8-a5913e3e3a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:16.372436 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effe6e1-0e9c-4e2a-a1e8-a5913e3e3a38 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:16.390447 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 578/1147] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:25.800942 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=22 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:26.388194 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=66 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:36.146944 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06385213-e792-4031-8fa5-447851923c5f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:36.147500 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06385213-e792-4031-8fa5-447851923c5f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:36.147500 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06385213-e792-4031-8fa5-447851923c5f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:36.148920 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 570/1148] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:36.154383 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c174e-62c8-4f09-a80d-17a41acd2ed1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:36.154791 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c174e-62c8-4f09-a80d-17a41acd2ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:36.154932 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c174e-62c8-4f09-a80d-17a41acd2ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:36.220427 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-838c174e-62c8-4f09-a80d-17a41acd2ed1 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:36.221576 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-838c174e-62c8-4f09-a80d-17a41acd2ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:36.315738 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 579/1149] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:36.615192 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd1ea-40fd-4533-9ac4-4be021e3d2f2 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:36.615472 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd1ea-40fd-4533-9ac4-4be021e3d2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:36.615743 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd1ea-40fd-4533-9ac4-4be021e3d2f2 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:36.617107 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 571/1150] 10.222.0.15 () {54 vars in 962 bytes} [Wed Nov 2 07:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:36.622012 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bef31-1c1f-4c06-895b-90e7fdec0de9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:36.622301 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bef31-1c1f-4c06-895b-90e7fdec0de9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:36.622578 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bef31-1c1f-4c06-895b-90e7fdec0de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:36.698508 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-df3bef31-1c1f-4c06-895b-90e7fdec0de9 None None] MFA Rules not processed for user `7a5b837bc9c04fc5a53077d152f4d6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:36.699718 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-df3bef31-1c1f-4c06-895b-90e7fdec0de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:36.720061 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:36.776743 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 580/1151] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Nov 2 07:21:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:36.785147 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecd1ea-40fd-4533-9ac4-4be021e3d2f2 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:36.788415 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0eecd1ea-40fd-4533-9ac4-4be021e3d2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:36.891174 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecd1ea-40fd-4533-9ac4-4be021e3d2f2 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:36.963921 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a53336-e560-466f-93d2-e6ae13cdb2d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a5b837bc9c04fc5a53077d152f4d6d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:36.964516 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53336-e560-466f-93d2-e6ae13cdb2d3 service glance] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:36.964827 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53336-e560-466f-93d2-e6ae13cdb2d3 service glance] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:36.965093 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53336-e560-466f-93d2-e6ae13cdb2d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:36.966653 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d0a53336-e560-466f-93d2-e6ae13cdb2d3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:37.018669 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:37.073806 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a53336-e560-466f-93d2-e6ae13cdb2d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:37.075230 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a53336-e560-466f-93d2-e6ae13cdb2d3 service glance] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:37.094006 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 572/1152] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:39.336575 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57f07e-ae0d-429c-b8d5-aa234fc4a7ef None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:39.337333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57f07e-ae0d-429c-b8d5-aa234fc4a7ef None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:39.337333 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57f07e-ae0d-429c-b8d5-aa234fc4a7ef None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:39.339720 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 581/1153] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:21:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:39.346114 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd7b52-781f-422b-9f39-8564672e7a17 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:39.346602 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd7b52-781f-422b-9f39-8564672e7a17 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:39.346717 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd7b52-781f-422b-9f39-8564672e7a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:39.414020 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-99bd7b52-781f-422b-9f39-8564672e7a17 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:39.415231 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-99bd7b52-781f-422b-9f39-8564672e7a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:39.511711 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 573/1154] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:39.781094 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57f07e-ae0d-429c-b8d5-aa234fc4a7ef None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:39.782563 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8f57f07e-ae0d-429c-b8d5-aa234fc4a7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:39.905701 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57f07e-ae0d-429c-b8d5-aa234fc4a7ef None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:39.953714 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:39.998213 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344a75e-f025-482c-93b9-d9a34b1ffccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:39.998814 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a75e-f025-482c-93b9-d9a34b1ffccf admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:39.999088 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a75e-f025-482c-93b9-d9a34b1ffccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:39.999356 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a75e-f025-482c-93b9-d9a34b1ffccf admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:39.999974 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344a75e-f025-482c-93b9-d9a34b1ffccf admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:40.001119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-b344a75e-f025-482c-93b9-d9a34b1ffccf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:21:40.001119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Nov 02 07:21:40.001119 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: Nov 02 07:21:40.002231 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344a75e-f025-482c-93b9-d9a34b1ffccf admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:40.020887 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 582/1155] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Nov 2 07:21:39 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:21:42.165699 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf147-9ac5-450a-88ef-8a2911f87c35 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:42.166274 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf147-9ac5-450a-88ef-8a2911f87c35 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:42.166274 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf147-9ac5-450a-88ef-8a2911f87c35 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:42.167554 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 574/1156] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:42.172864 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4acfb-239c-4315-8aec-e405dc204564 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:42.173316 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4acfb-239c-4315-8aec-e405dc204564 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:42.173498 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4acfb-239c-4315-8aec-e405dc204564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:42.241396 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7af4acfb-239c-4315-8aec-e405dc204564 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:42.242568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7af4acfb-239c-4315-8aec-e405dc204564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:42.342233 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 583/1157] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:42.634226 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccf147-9ac5-450a-88ef-8a2911f87c35 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:42.635965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-62ccf147-9ac5-450a-88ef-8a2911f87c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:42.648359 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:42.784152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccf147-9ac5-450a-88ef-8a2911f87c35 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:42.864166 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:42.895937 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36472888-1af5-4551-b4df-b4f396410e1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00ab674b018e474ab363e8e17d225e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:42.896561 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472888-1af5-4551-b4df-b4f396410e1b service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:42.896839 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472888-1af5-4551-b4df-b4f396410e1b service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:42.897107 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472888-1af5-4551-b4df-b4f396410e1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:42.898680 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-36472888-1af5-4551-b4df-b4f396410e1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:42.986307 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36472888-1af5-4551-b4df-b4f396410e1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:42.987869 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36472888-1af5-4551-b4df-b4f396410e1b service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:43.005660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 575/1158] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:45.301491 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1c06d-6c3e-4d9f-baf8-0ae8cceff881 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:45.302030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1c06d-6c3e-4d9f-baf8-0ae8cceff881 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:45.302030 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1c06d-6c3e-4d9f-baf8-0ae8cceff881 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:45.303367 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 584/1159] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:45.308579 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9f757-ee4c-4b09-8de8-f905a4e58bd2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:45.309005 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9f757-ee4c-4b09-8de8-f905a4e58bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:45.309120 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9f757-ee4c-4b09-8de8-f905a4e58bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:45.374536 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-42a9f757-ee4c-4b09-8de8-f905a4e58bd2 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:45.375724 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-42a9f757-ee4c-4b09-8de8-f905a4e58bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:45.421169 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:45.471327 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 576/1160] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:45.755402 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1c06d-6c3e-4d9f-baf8-0ae8cceff881 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:45.756978 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-eba1c06d-6c3e-4d9f-baf8-0ae8cceff881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:45.780598 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:45.858846 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1c06d-6c3e-4d9f-baf8-0ae8cceff881 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:45.931301 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d3581-b51c-4985-85ab-eff3021a7793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:45.931824 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d3581-b51c-4985-85ab-eff3021a7793 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:45.932102 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d3581-b51c-4985-85ab-eff3021a7793 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:45.932366 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d3581-b51c-4985-85ab-eff3021a7793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:45.933963 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e87d3581-b51c-4985-85ab-eff3021a7793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:46.006661 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:46.042287 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d3581-b51c-4985-85ab-eff3021a7793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:46.043719 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d3581-b51c-4985-85ab-eff3021a7793 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:46.061446 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 585/1161] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:48.249965 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ff69c-b566-49f6-a657-c5c8920620c9 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:48.250483 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ff69c-b566-49f6-a657-c5c8920620c9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:48.250483 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ff69c-b566-49f6-a657-c5c8920620c9 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:48.252090 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 577/1162] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:48.257464 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd50611-579c-452d-989e-d8fa0f519838 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:48.257891 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd50611-579c-452d-989e-d8fa0f519838 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:48.257999 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd50611-579c-452d-989e-d8fa0f519838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:48.323187 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8fd50611-579c-452d-989e-d8fa0f519838 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:48.324393 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8fd50611-579c-452d-989e-d8fa0f519838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:48.420115 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 586/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:48.704698 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ff69c-b566-49f6-a657-c5c8920620c9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:48.706218 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7c2ff69c-b566-49f6-a657-c5c8920620c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:48.807498 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ff69c-b566-49f6-a657-c5c8920620c9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:48.877559 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:48.879818 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e4afc-d35a-49a8-8b53-b50a336dd3ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:48.880460 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e4afc-d35a-49a8-8b53-b50a336dd3ce service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:48.880960 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e4afc-d35a-49a8-8b53-b50a336dd3ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:48.881388 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e4afc-d35a-49a8-8b53-b50a336dd3ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:48.883133 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-764e4afc-d35a-49a8-8b53-b50a336dd3ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:48.992616 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e4afc-d35a-49a8-8b53-b50a336dd3ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:48.993959 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e4afc-d35a-49a8-8b53-b50a336dd3ce service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:49.012296 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 578/1164] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:51.251783 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7174133-faa2-4c87-9a31-7c1342666474 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:51.252352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7174133-faa2-4c87-9a31-7c1342666474 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:51.252352 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7174133-faa2-4c87-9a31-7c1342666474 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:51.253740 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 587/1165] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:51.259574 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d30da9-f43d-4f81-abb3-f2d28b6bde7b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:51.259997 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d30da9-f43d-4f81-abb3-f2d28b6bde7b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:51.260145 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d30da9-f43d-4f81-abb3-f2d28b6bde7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:51.324764 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e2d30da9-f43d-4f81-abb3-f2d28b6bde7b None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:51.325966 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e2d30da9-f43d-4f81-abb3-f2d28b6bde7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:51.379203 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:51.423523 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 579/1166] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:51.711453 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7174133-faa2-4c87-9a31-7c1342666474 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:51.713093 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a7174133-faa2-4c87-9a31-7c1342666474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:51.748725 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:51.815806 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7174133-faa2-4c87-9a31-7c1342666474 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:51.889921 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee8eed-290b-4818-a644-31bc3bbc3c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:51.890531 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee8eed-290b-4818-a644-31bc3bbc3c8e service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:51.890858 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee8eed-290b-4818-a644-31bc3bbc3c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:51.891073 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee8eed-290b-4818-a644-31bc3bbc3c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:51.892683 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9fee8eed-290b-4818-a644-31bc3bbc3c8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:51.971481 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:52.000422 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee8eed-290b-4818-a644-31bc3bbc3c8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:52.002011 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee8eed-290b-4818-a644-31bc3bbc3c8e service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:52.021489 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 588/1167] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:54.390532 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd644-d7cd-4fef-9a78-240eedd6011d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:54.391068 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd644-d7cd-4fef-9a78-240eedd6011d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:54.391175 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd644-d7cd-4fef-9a78-240eedd6011d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:54.392446 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 580/1168] 10.222.0.15 () {54 vars in 927 bytes} [Wed Nov 2 07:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:54.398150 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1245be2-1d95-4c1b-9582-17d9eed09f5e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:54.398538 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1245be2-1d95-4c1b-9582-17d9eed09f5e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:54.398701 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1245be2-1d95-4c1b-9582-17d9eed09f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:54.463510 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a1245be2-1d95-4c1b-9582-17d9eed09f5e None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:54.464811 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a1245be2-1d95-4c1b-9582-17d9eed09f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:54.559562 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 589/1169] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:54.849233 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fd644-d7cd-4fef-9a78-240eedd6011d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:54.850769 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5a7fd644-d7cd-4fef-9a78-240eedd6011d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:54.953738 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fd644-d7cd-4fef-9a78-240eedd6011d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:55.027446 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaaf21b-325d-406a-bc1b-617099db0c51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:55.028052 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaf21b-325d-406a-bc1b-617099db0c51 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:55.028338 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaf21b-325d-406a-bc1b-617099db0c51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:55.028634 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaf21b-325d-406a-bc1b-617099db0c51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:55.030223 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1aaaf21b-325d-406a-bc1b-617099db0c51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:55.039278 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:55.139197 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaaf21b-325d-406a-bc1b-617099db0c51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:55.140650 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaaf21b-325d-406a-bc1b-617099db0c51 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:55.159025 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 581/1170] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:04.558067 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=82 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:05.156793 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=50 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:14.927495 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fe8ba-01bf-4aaa-b58d-5332549cb005 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:14.928038 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fe8ba-01bf-4aaa-b58d-5332549cb005 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:14.928038 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fe8ba-01bf-4aaa-b58d-5332549cb005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:15.002306 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-102fe8ba-01bf-4aaa-b58d-5332549cb005 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:15.003572 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-102fe8ba-01bf-4aaa-b58d-5332549cb005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:15.099354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 590/1171] 10.222.0.15 () {58 vars in 958 bytes} [Wed Nov 2 07:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:15.107379 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6310ce5-4e1e-40c6-8b7b-b1594b151693 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:15.107865 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6310ce5-4e1e-40c6-8b7b-b1594b151693 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:15.107969 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6310ce5-4e1e-40c6-8b7b-b1594b151693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:15.183449 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a6310ce5-4e1e-40c6-8b7b-b1594b151693 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:15.184605 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a6310ce5-4e1e-40c6-8b7b-b1594b151693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:15.279300 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 582/1172] 10.222.0.15 () {58 vars in 958 bytes} [Wed Nov 2 07:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:15.284097 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fe8ba-01bf-4aaa-b58d-5332549cb005 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:15.285847 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-102fe8ba-01bf-4aaa-b58d-5332549cb005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:15.297167 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fe8ba-01bf-4aaa-b58d-5332549cb005 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:15.358545 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:15.391881 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14240d28-0b38-49d2-a8e7-e5b841919b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:15.392516 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14240d28-0b38-49d2-a8e7-e5b841919b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:15.392829 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14240d28-0b38-49d2-a8e7-e5b841919b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:15.393103 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14240d28-0b38-49d2-a8e7-e5b841919b4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:15.394270 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14240d28-0b38-49d2-a8e7-e5b841919b4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:23:15.394568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14240d28-0b38-49d2-a8e7-e5b841919b4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:15.395571 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-14240d28-0b38-49d2-a8e7-e5b841919b4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:23:15.395571 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Nov 02 07:23:15.395571 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: Nov 02 07:23:15.396452 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14240d28-0b38-49d2-a8e7-e5b841919b4f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:15.407483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 591/1173] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Nov 2 07:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:23:15.417100 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6310ce5-4e1e-40c6-8b7b-b1594b151693 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:15.426605 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6310ce5-4e1e-40c6-8b7b-b1594b151693 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:15.488241 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:15.521548 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e307-b52f-4d6a-9605-ee8899374dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:15.522152 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e307-b52f-4d6a-9605-ee8899374dbf admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:15.522426 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e307-b52f-4d6a-9605-ee8899374dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:15.522693 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e307-b52f-4d6a-9605-ee8899374dbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:15.523311 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821e307-b52f-4d6a-9605-ee8899374dbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:15.524318 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-2821e307-b52f-4d6a-9605-ee8899374dbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:23:15.524318 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Nov 02 07:23:15.524318 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: Nov 02 07:23:15.525526 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821e307-b52f-4d6a-9605-ee8899374dbf admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:15.579572 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 583/1174] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Nov 2 07:23:15 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:23:15.583655 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14240d28-0b38-49d2-a8e7-e5b841919b4f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:15.585178 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-14240d28-0b38-49d2-a8e7-e5b841919b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:15.617941 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14240d28-0b38-49d2-a8e7-e5b841919b4f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:15.711347 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f6dfa-00bf-4519-9583-1de2da6dffae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:15.711952 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f6dfa-00bf-4519-9583-1de2da6dffae admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:15.712228 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f6dfa-00bf-4519-9583-1de2da6dffae admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:15.712499 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f6dfa-00bf-4519-9583-1de2da6dffae admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:15.713161 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f6dfa-00bf-4519-9583-1de2da6dffae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:15.714157 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-149f6dfa-00bf-4519-9583-1de2da6dffae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:23:15.714157 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Nov 02 07:23:15.714157 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: Nov 02 07:23:15.715354 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f6dfa-00bf-4519-9583-1de2da6dffae admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:15.758774 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 592/1175] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Nov 2 07:23:15 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:23:15.762783 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e307-b52f-4d6a-9605-ee8899374dbf admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:15.764289 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2821e307-b52f-4d6a-9605-ee8899374dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:15.825348 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e307-b52f-4d6a-9605-ee8899374dbf admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:15.908063 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:15.920744 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd82ab8-db4b-4b5d-8356-11c99bdaeb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:15.921351 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd82ab8-db4b-4b5d-8356-11c99bdaeb71 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:15.921630 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd82ab8-db4b-4b5d-8356-11c99bdaeb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:15.921898 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd82ab8-db4b-4b5d-8356-11c99bdaeb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:15.922530 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd82ab8-db4b-4b5d-8356-11c99bdaeb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:15.924164 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd82ab8-db4b-4b5d-8356-11c99bdaeb71 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:15.937094 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 584/1176] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Nov 2 07:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:23:15.941099 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f6dfa-00bf-4519-9583-1de2da6dffae admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:15.942601 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-149f6dfa-00bf-4519-9583-1de2da6dffae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:15.953786 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f6dfa-00bf-4519-9583-1de2da6dffae admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:15.979007 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:16.049230 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea575a59-9425-4eb5-8d70-223191106011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:16.049903 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575a59-9425-4eb5-8d70-223191106011 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:16.050707 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575a59-9425-4eb5-8d70-223191106011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:16.051005 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575a59-9425-4eb5-8d70-223191106011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:16.051637 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea575a59-9425-4eb5-8d70-223191106011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:16.053255 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea575a59-9425-4eb5-8d70-223191106011 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:16.065483 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 593/1177] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Nov 2 07:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:23:16.069514 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd82ab8-db4b-4b5d-8356-11c99bdaeb71 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:16.071045 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9dd82ab8-db4b-4b5d-8356-11c99bdaeb71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:16.103992 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd82ab8-db4b-4b5d-8356-11c99bdaeb71 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:16.195660 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e526b44-0951-4d6b-99cb-c882ab80ae2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:16.196305 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e526b44-0951-4d6b-99cb-c882ab80ae2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:16.196586 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e526b44-0951-4d6b-99cb-c882ab80ae2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:16.196865 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e526b44-0951-4d6b-99cb-c882ab80ae2d admin admin] PATH_INFO: `/v3/projects/541eaea8faf344fc8f3277f67f368ca7/users/ee22805747fc4253a310adf0051e0f6a/roles/6914eb62761341d7b8640a878225f115` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:16.219353 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e526b44-0951-4d6b-99cb-c882ab80ae2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=541eaea8faf344fc8f3277f67f368ca7, user_id=ee22805747fc4253a310adf0051e0f6a, role_id=6914eb62761341d7b8640a878225f115)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:16.222640 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e526b44-0951-4d6b-99cb-c882ab80ae2d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:16.246364 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 585/1178] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Nov 2 07:23:16 2022] PUT /identity/v3/projects/541eaea8faf344fc8f3277f67f368ca7/users/ee22805747fc4253a310adf0051e0f6a/roles/6914eb62761341d7b8640a878225f115 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:23:16.257473 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a0bc3-88e1-45d8-8f03-920725da3cd7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:16.257846 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a0bc3-88e1-45d8-8f03-920725da3cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:16.258138 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a0bc3-88e1-45d8-8f03-920725da3cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:16.314823 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-705a0bc3-88e1-45d8-8f03-920725da3cd7 None None] MFA Rules not processed for user `ee22805747fc4253a310adf0051e0f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:16.316556 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-705a0bc3-88e1-45d8-8f03-920725da3cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:16.394344 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 594/1179] 10.222.0.15 () {58 vars in 958 bytes} [Wed Nov 2 07:23:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:16.407255 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36187e8-252f-498b-803a-10b83f63d193 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:16.407540 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36187e8-252f-498b-803a-10b83f63d193 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:16.407824 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36187e8-252f-498b-803a-10b83f63d193 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:16.411921 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 586/1180] 10.222.0.15 () {54 vars in 963 bytes} [Wed Nov 2 07:23:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:16.414039 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d5de4-a7eb-42ba-bd2f-1594fb0c54f7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:16.414326 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d5de4-a7eb-42ba-bd2f-1594fb0c54f7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:16.414632 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d5de4-a7eb-42ba-bd2f-1594fb0c54f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:16.446938 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:16.491867 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e40d5de4-a7eb-42ba-bd2f-1594fb0c54f7 None None] MFA Rules not processed for user `e0f9d9dc1f7143e0be709e17d1a5f484`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:16.493091 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e40d5de4-a7eb-42ba-bd2f-1594fb0c54f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:16.567135 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 595/1181] 10.222.0.15 () {58 vars in 1060 bytes} [Wed Nov 2 07:23:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:16.574741 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36187e8-252f-498b-803a-10b83f63d193 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:16.576237 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a36187e8-252f-498b-803a-10b83f63d193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:16.625822 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:16.681949 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36187e8-252f-498b-803a-10b83f63d193 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:16.755109 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c40b49-68ac-4be1-8902-0d20421e315f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0f9d9dc1f7143e0be709e17d1a5f484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:16.755712 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c40b49-68ac-4be1-8902-0d20421e315f service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:16.755989 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c40b49-68ac-4be1-8902-0d20421e315f service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:16.756258 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c40b49-68ac-4be1-8902-0d20421e315f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:16.757872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-21c40b49-68ac-4be1-8902-0d20421e315f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:16.855457 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c40b49-68ac-4be1-8902-0d20421e315f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:16.856872 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c40b49-68ac-4be1-8902-0d20421e315f service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:16.860278 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:16.875565 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 587/1182] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:16.920748 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2471f7-b9e9-40e7-912f-8a44dc927d25 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:16.921034 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2471f7-b9e9-40e7-912f-8a44dc927d25 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:16.921310 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2471f7-b9e9-40e7-912f-8a44dc927d25 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:16.922568 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 596/1183] 10.222.0.15 () {52 vars in 830 bytes} [Wed Nov 2 07:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:16.926019 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c40b49-68ac-4be1-8902-0d20421e315f service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:16.927212 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-21c40b49-68ac-4be1-8902-0d20421e315f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:17.024946 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c40b49-68ac-4be1-8902-0d20421e315f service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:17.117118 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:17.117680 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:17.117956 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:17.118226 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] PATH_INFO: `/v3/users/ee22805747fc4253a310adf0051e0f6a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:17.130635 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee22805747fc4253a310adf0051e0f6a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:17.131654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:23:17.131654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Nov 02 07:23:17.131654 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: Nov 02 07:23:17.132902 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:17.142244 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:17.196957 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee22805747fc4253a310adf0051e0f6a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:23:17.197590 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee22805747fc4253a310adf0051e0f6a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:23:17.197895 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee22805747fc4253a310adf0051e0f6a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:23:17.207245 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee22805747fc4253a310adf0051e0f6a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:23:17.231100 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee22805747fc4253a310adf0051e0f6a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:23:17.249893 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 588/1184] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Nov 2 07:23:16 2022] DELETE /identity/v3/users/ee22805747fc4253a310adf0051e0f6a => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:23:17.254747 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336334f-377c-4394-ba8e-9b8b305d3022 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:17.255224 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336334f-377c-4394-ba8e-9b8b305d3022 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:17.255320 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336334f-377c-4394-ba8e-9b8b305d3022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:17.311792 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a336334f-377c-4394-ba8e-9b8b305d3022 None None] MFA Rules not processed for user `6ac58377d56b4184a21764af3c195e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:17.313075 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a336334f-377c-4394-ba8e-9b8b305d3022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:17.389280 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:17.404342 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 597/1185] 10.222.0.15 () {58 vars in 958 bytes} [Wed Nov 2 07:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:17.414904 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:17.416174 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:17.524123 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a83bf7-0314-4959-8772-e86220e2cb89 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:17.576603 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:17.597500 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0849e-d400-45cc-bc5b-17665455add3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36450acd551f40d08a4bc21c9a9c8877', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41551cd452f845849527d7c06ff6e719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:17.598112 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0849e-d400-45cc-bc5b-17665455add3 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:17.598391 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0849e-d400-45cc-bc5b-17665455add3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:17.598663 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0849e-d400-45cc-bc5b-17665455add3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:17.600258 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-54a0849e-d400-45cc-bc5b-17665455add3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:17.710861 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0849e-d400-45cc-bc5b-17665455add3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:17.712360 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0849e-d400-45cc-bc5b-17665455add3 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:17.729970 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 589/1186] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:18.073123 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336334f-377c-4394-ba8e-9b8b305d3022 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:18.074358 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a336334f-377c-4394-ba8e-9b8b305d3022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:18.087385 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336334f-377c-4394-ba8e-9b8b305d3022 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:18.182411 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ac58377d56b4184a21764af3c195e68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316fe9e17d5440ea82f955c062d5fbe0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:18.182965 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:18.183239 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:18.183505 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] PATH_INFO: `/v3/projects/541eaea8faf344fc8f3277f67f368ca7` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:18.193254 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] RBAC: Authorizing `identity:delete_project(project_id=541eaea8faf344fc8f3277f67f368ca7)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:18.194262 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:23:18.194262 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Nov 02 07:23:18.194262 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: Nov 02 07:23:18.195451 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:18.253378 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '541eaea8faf344fc8f3277f67f368ca7', 'request_id': 'req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:23:18.266703 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '541eaea8faf344fc8f3277f67f368ca7', 'request_id': 'req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:23:18.276246 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '541eaea8faf344fc8f3277f67f368ca7', 'request_id': 'req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:23:18.300970 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] The token cache is being invalidate because project 541eaea8faf344fc8f3277f67f368ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:23:18.301249 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:23:18.301767 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcff6d95-72ef-4ab8-9952-a7d2b0b4a38e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:23:18.303116 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 598/1187] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Nov 2 07:23:18 2022] DELETE /identity/v3/projects/541eaea8faf344fc8f3277f67f368ca7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:23:27.727674 dv-cinder-804035-51-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=62 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:28.298363 dv-cinder-804035-51-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}